2024-05-03T20:17:38.313 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-03T20:17:38.313 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/kernel_untar_build}} --owner scheduled_rishabh@teuthology --verbose --name rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687487 -- /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687487/orig.config.yaml 2024-05-03T20:17:38.371 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-03T20:17:38.460 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687487 branch: wip-rishabh-testing-20240501.193033 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/kernel_untar_build}} email: null first_in_suite: false job_id: '7687487' kernel: client: sha1: distro flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240501.193033 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240501.193033 sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1698 sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea sleep_before_teardown: 0 subset: 434/512 suite: fs suite_branch: wip-rishabh-testing-20240501.193033 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_9a047fe7599a549042d06a3fb5b6813a203eabea/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi181.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMuJgBR7dwrCRXcE0MRztbywZDTsTI2MxthXnWFQRLNxRkd7MlDYH2IdXqh1FRIkM/xEhx/Uyhvd2vNgohg2qis= smithi196.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBpeNGjmmxDguhY9ZoRg+7WOENxhsqR+2Nuq7fIgvxM6U9pn89v6rJgKvx8qlF1a49espqlHJFgKCv9lH4UxUao= smithi204.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDudUQ414Z11W7NN34BCo5zkmQMHnmmVNIFbg4FQiR/9i+xblCUsfXiZrHQYSKYjkelN4IHaxmisAWd/hbohSws= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - quiescer: null - workunit: clients: all: - kernel_untar_build.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-03_13:56:19 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-03T20:17:38.460 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_9a047fe7599a549042d06a3fb5b6813a203eabea/qa; will attempt to use it 2024-05-03T20:17:38.461 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_9a047fe7599a549042d06a3fb5b6813a203eabea/qa/tasks 2024-05-03T20:17:38.461 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-03T20:17:38.462 INFO:teuthology.task.internal:Checking packages... 2024-05-03T20:17:38.484 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '9a047fe7599a549042d06a3fb5b6813a203eabea' 2024-05-03T20:17:38.484 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-03T20:17:38.485 INFO:teuthology.packaging:ref: None 2024-05-03T20:17:38.485 INFO:teuthology.packaging:tag: None 2024-05-03T20:17:38.485 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240501.193033 2024-05-03T20:17:38.485 INFO:teuthology.packaging:sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T20:17:38.485 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240501.193033 2024-05-03T20:17:38.750 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3450.g9a047fe7 2024-05-03T20:17:38.752 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-03T20:17:38.758 INFO:teuthology.task.internal:no buildpackages task found 2024-05-03T20:17:38.758 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-03T20:17:38.806 INFO:teuthology.task.internal:Saving configuration 2024-05-03T20:17:38.825 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-03T20:17:38.829 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-03T20:17:38.856 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687487', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-03 20:10:28.696450', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMuJgBR7dwrCRXcE0MRztbywZDTsTI2MxthXnWFQRLNxRkd7MlDYH2IdXqh1FRIkM/xEhx/Uyhvd2vNgohg2qis='} 2024-05-03T20:17:38.876 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687487', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-03 20:10:28.693928', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBpeNGjmmxDguhY9ZoRg+7WOENxhsqR+2Nuq7fIgvxM6U9pn89v6rJgKvx8qlF1a49espqlHJFgKCv9lH4UxUao='} 2024-05-03T20:17:38.896 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687487', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-03 20:10:28.695322', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDudUQ414Z11W7NN34BCo5zkmQMHnmmVNIFbg4FQiR/9i+xblCUsfXiZrHQYSKYjkelN4IHaxmisAWd/hbohSws='} 2024-05-03T20:17:38.897 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-03T20:17:38.902 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-03T20:17:38.902 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-03T20:17:38.902 INFO:teuthology.task.internal:roles: ubuntu@smithi204.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-03T20:17:38.902 INFO:teuthology.run_tasks:Running task console_log... 2024-05-03T20:17:39.019 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb412abd160>, signals=[15]) 2024-05-03T20:17:39.020 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-03T20:17:39.026 INFO:teuthology.task.internal:Opening connections... 2024-05-03T20:17:39.026 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi181.front.sepia.ceph.com 2024-05-03T20:17:39.028 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T20:17:39.107 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi196.front.sepia.ceph.com 2024-05-03T20:17:39.108 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T20:17:39.185 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi204.front.sepia.ceph.com 2024-05-03T20:17:39.186 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T20:17:39.262 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-03T20:17:39.269 DEBUG:teuthology.orchestra.run.smithi181:> uname -m 2024-05-03T20:17:39.288 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2024-05-03T20:17:39.288 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2024-05-03T20:17:39.345 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="CentOS Stream" 2024-05-03T20:17:39.345 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="9" 2024-05-03T20:17:39.345 INFO:teuthology.orchestra.run.smithi181.stdout:ID="centos" 2024-05-03T20:17:39.346 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="rhel fedora" 2024-05-03T20:17:39.346 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="9" 2024-05-03T20:17:39.346 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el9" 2024-05-03T20:17:39.346 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-03T20:17:39.346 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2024-05-03T20:17:39.346 INFO:teuthology.orchestra.run.smithi181.stdout:LOGO="fedora-logo-icon" 2024-05-03T20:17:39.346 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-03T20:17:39.346 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://centos.org/" 2024-05-03T20:17:39.346 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-03T20:17:39.346 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-03T20:17:39.346 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-03T20:17:39.347 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2024-05-03T20:17:39.370 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-05-03T20:17:39.388 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-05-03T20:17:39.389 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-05-03T20:17:39.445 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Stream" 2024-05-03T20:17:39.445 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="9" 2024-05-03T20:17:39.445 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos" 2024-05-03T20:17:39.445 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora" 2024-05-03T20:17:39.445 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="9" 2024-05-03T20:17:39.445 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el9" 2024-05-03T20:17:39.445 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-03T20:17:39.445 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-05-03T20:17:39.445 INFO:teuthology.orchestra.run.smithi196.stdout:LOGO="fedora-logo-icon" 2024-05-03T20:17:39.446 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-03T20:17:39.446 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/" 2024-05-03T20:17:39.446 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-03T20:17:39.446 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-03T20:17:39.446 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-03T20:17:39.446 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-05-03T20:17:39.470 DEBUG:teuthology.orchestra.run.smithi204:> uname -m 2024-05-03T20:17:39.487 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2024-05-03T20:17:39.487 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2024-05-03T20:17:39.545 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="CentOS Stream" 2024-05-03T20:17:39.545 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="9" 2024-05-03T20:17:39.545 INFO:teuthology.orchestra.run.smithi204.stdout:ID="centos" 2024-05-03T20:17:39.545 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE="rhel fedora" 2024-05-03T20:17:39.545 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="9" 2024-05-03T20:17:39.545 INFO:teuthology.orchestra.run.smithi204.stdout:PLATFORM_ID="platform:el9" 2024-05-03T20:17:39.545 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-03T20:17:39.545 INFO:teuthology.orchestra.run.smithi204.stdout:ANSI_COLOR="0;31" 2024-05-03T20:17:39.545 INFO:teuthology.orchestra.run.smithi204.stdout:LOGO="fedora-logo-icon" 2024-05-03T20:17:39.545 INFO:teuthology.orchestra.run.smithi204.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-03T20:17:39.545 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://centos.org/" 2024-05-03T20:17:39.546 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-03T20:17:39.546 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-03T20:17:39.546 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-03T20:17:39.546 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2024-05-03T20:17:39.570 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-03T20:17:39.580 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-03T20:17:39.597 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-03T20:17:39.598 DEBUG:teuthology.orchestra.run.smithi181:> test '!' -e /home/ubuntu/cephtest 2024-05-03T20:17:39.600 DEBUG:teuthology.orchestra.run.smithi196:> test '!' -e /home/ubuntu/cephtest 2024-05-03T20:17:39.602 DEBUG:teuthology.orchestra.run.smithi204:> test '!' -e /home/ubuntu/cephtest 2024-05-03T20:17:39.618 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-03T20:17:39.624 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-03T20:17:39.624 DEBUG:teuthology.orchestra.run.smithi181:> test -z $(ls -A /var/lib/ceph) 2024-05-03T20:17:39.656 DEBUG:teuthology.orchestra.run.smithi196:> test -z $(ls -A /var/lib/ceph) 2024-05-03T20:17:39.658 DEBUG:teuthology.orchestra.run.smithi204:> test -z $(ls -A /var/lib/ceph) 2024-05-03T20:17:39.759 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-03T20:17:39.836 INFO:teuthology.run_tasks:Running task kernel... 2024-05-03T20:17:39.850 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-05-03T20:17:39.851 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-03T20:17:39.851 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-03T20:17:39.851 DEBUG:teuthology.orchestra.run.smithi181:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-03T20:17:39.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T20:17:39.867 DEBUG:teuthology.orchestra.run.smithi181:> uname -r 2024-05-03T20:17:39.925 INFO:teuthology.orchestra.run.smithi181.stdout:5.14.0-437.el9.x86_64 2024-05-03T20:17:39.925 INFO:teuthology.task.kernel:Running kernel on smithi181: 5.14.0-437.el9.x86_64 2024-05-03T20:17:39.925 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum install -y kernel 2024-05-03T20:17:42.236 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-05-03T20:17:44.087 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-05-03T20:17:49.464 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.2 MB 00:01 2024-05-03T20:17:51.164 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 16 kB 00:00 2024-05-03T20:17:53.157 INFO:teuthology.orchestra.run.smithi181.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-05-03T20:17:58.348 INFO:teuthology.orchestra.run.smithi181.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-05-03T20:18:00.213 INFO:teuthology.orchestra.run.smithi181.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-03T20:18:00.213 INFO:teuthology.orchestra.run.smithi181.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-03T20:18:00.303 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-05-03T20:18:00.305 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2024-05-03T20:18:00.305 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-05-03T20:18:00.412 DEBUG:teuthology.orchestra.run.smithi181:> echo no | sudo yum reinstall kernel || true 2024-05-03T20:18:00.912 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:00:02 ago on Fri 03 May 2024 08:17:58 PM UTC. 2024-05-03T20:18:01.031 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-05-03T20:18:01.046 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-05-03T20:18:01.046 INFO:teuthology.orchestra.run.smithi181.stdout: Package Architecture Version Repository Size 2024-05-03T20:18:01.046 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-05-03T20:18:01.046 INFO:teuthology.orchestra.run.smithi181.stdout:Reinstalling: 2024-05-03T20:18:01.046 INFO:teuthology.orchestra.run.smithi181.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-05-03T20:18:01.046 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:18:01.046 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-05-03T20:18:01.046 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-05-03T20:18:01.047 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:18:01.047 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 255 k 2024-05-03T20:18:01.047 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 0 2024-05-03T20:18:01.047 INFO:teuthology.orchestra.run.smithi181.stderr:Operation aborted. 2024-05-03T20:18:01.103 INFO:teuthology.orchestra.run.smithi181.stdout:Is this ok [y/N]: 2024-05-03T20:18:01.103 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum reinstall -y kernel || true 2024-05-03T20:18:01.596 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:00:03 ago on Fri 03 May 2024 08:17:58 PM UTC. 2024-05-03T20:18:01.713 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-05-03T20:18:01.713 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-05-03T20:18:01.713 INFO:teuthology.orchestra.run.smithi181.stdout: Package Architecture Version Repository Size 2024-05-03T20:18:01.713 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-05-03T20:18:01.714 INFO:teuthology.orchestra.run.smithi181.stdout:Reinstalling: 2024-05-03T20:18:01.714 INFO:teuthology.orchestra.run.smithi181.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-05-03T20:18:01.714 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:18:01.714 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-05-03T20:18:01.714 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-05-03T20:18:01.714 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:18:01.714 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 255 k 2024-05-03T20:18:01.715 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 0 2024-05-03T20:18:01.715 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-05-03T20:18:02.100 INFO:teuthology.orchestra.run.smithi181.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.6 MB/s | 255 kB 00:00 2024-05-03T20:18:02.100 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-05-03T20:18:02.100 INFO:teuthology.orchestra.run.smithi181.stdout:Total 660 kB/s | 255 kB 00:00 2024-05-03T20:18:02.119 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-05-03T20:18:02.293 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-05-03T20:18:02.293 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-05-03T20:18:02.298 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-05-03T20:18:02.299 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-05-03T20:18:02.438 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-05-03T20:18:02.509 INFO:teuthology.orchestra.run.smithi181.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-05-03T20:18:02.510 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-05-03T20:18:03.177 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-05-03T20:18:03.177 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-05-03T20:18:03.878 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-05-03T20:18:03.878 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:18:03.878 INFO:teuthology.orchestra.run.smithi181.stdout:Reinstalled: 2024-05-03T20:18:03.878 INFO:teuthology.orchestra.run.smithi181.stdout: kernel-5.14.0-437.el9.x86_64 2024-05-03T20:18:03.878 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:18:03.878 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-05-03T20:18:04.071 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q kernel | sort -rV | head -n 1 2024-05-03T20:18:04.118 INFO:teuthology.orchestra.run.smithi181.stdout:kernel-5.14.0-437.el9.x86_64 2024-05-03T20:18:04.119 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-05-03T20:18:04.119 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-05-03T20:18:04.119 DEBUG:teuthology.parallel:result is None 2024-05-03T20:18:04.119 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-03T20:18:04.127 INFO:teuthology.task.internal:Creating test directory... 2024-05-03T20:18:04.128 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-03T20:18:04.130 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-03T20:18:04.133 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-03T20:18:04.150 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-03T20:18:04.177 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-03T20:18:04.236 INFO:teuthology.task.internal:Creating archive directory... 2024-05-03T20:18:04.236 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-03T20:18:04.238 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-03T20:18:04.241 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-03T20:18:04.409 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-03T20:18:04.417 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-03T20:18:04.417 DEBUG:teuthology.orchestra.run.smithi196:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-03T20:18:04.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T20:18:04.433 DEBUG:teuthology.orchestra.run.smithi204:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-03T20:18:04.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T20:18:04.450 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-03T20:18:04.453 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-03T20:18:04.476 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-03T20:18:04.478 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T20:18:04.506 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T20:18:04.513 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T20:18:04.524 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T20:18:04.532 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T20:18:04.550 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T20:18:04.552 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-03T20:18:04.559 INFO:teuthology.task.internal:Configuring sudo... 2024-05-03T20:18:04.559 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-03T20:18:04.561 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-03T20:18:04.568 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-03T20:18:04.618 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-03T20:18:04.627 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-03T20:18:04.627 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-03T20:18:04.630 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-03T20:18:04.634 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-03T20:18:04.675 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T20:18:04.719 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T20:18:04.771 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T20:18:04.852 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T20:18:04.890 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-03T20:18:04.890 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-03T20:18:04.953 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T20:18:04.996 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T20:18:05.042 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T20:18:05.125 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T20:18:05.166 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-03T20:18:05.167 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-03T20:18:05.232 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T20:18:05.277 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T20:18:05.334 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T20:18:05.417 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T20:18:05.459 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-03T20:18:05.459 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-03T20:18:05.525 DEBUG:teuthology.orchestra.run.smithi181:> sudo service rsyslog restart 2024-05-03T20:18:05.527 DEBUG:teuthology.orchestra.run.smithi196:> sudo service rsyslog restart 2024-05-03T20:18:05.529 DEBUG:teuthology.orchestra.run.smithi204:> sudo service rsyslog restart 2024-05-03T20:18:05.562 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T20:18:05.569 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T20:18:05.606 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T20:18:06.001 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-03T20:18:06.010 INFO:teuthology.task.internal:Starting timer... 2024-05-03T20:18:06.010 INFO:teuthology.run_tasks:Running task pcp... 2024-05-03T20:18:06.064 INFO:teuthology.run_tasks:Running task selinux... 2024-05-03T20:18:06.079 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-03T20:18:06.079 DEBUG:teuthology.orchestra.run.smithi181:> sudo service auditd rotate 2024-05-03T20:18:06.131 INFO:teuthology.orchestra.run.smithi181.stdout:Rotating logs: 2024-05-03T20:18:06.133 DEBUG:teuthology.orchestra.run.smithi196:> sudo service auditd rotate 2024-05-03T20:18:06.233 INFO:teuthology.orchestra.run.smithi196.stdout:Rotating logs: 2024-05-03T20:18:06.235 DEBUG:teuthology.orchestra.run.smithi204:> sudo service auditd rotate 2024-05-03T20:18:06.305 INFO:teuthology.orchestra.run.smithi204.stdout:Rotating logs: 2024-05-03T20:18:06.307 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-03T20:18:06.307 DEBUG:teuthology.orchestra.run.smithi181:> /usr/sbin/getenforce 2024-05-03T20:18:06.331 INFO:teuthology.orchestra.run.smithi181.stdout:Permissive 2024-05-03T20:18:06.331 DEBUG:teuthology.orchestra.run.smithi196:> /usr/sbin/getenforce 2024-05-03T20:18:06.358 INFO:teuthology.orchestra.run.smithi196.stdout:Permissive 2024-05-03T20:18:06.358 DEBUG:teuthology.orchestra.run.smithi204:> /usr/sbin/getenforce 2024-05-03T20:18:06.394 INFO:teuthology.orchestra.run.smithi204.stdout:Permissive 2024-05-03T20:18:06.394 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi181.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi196.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi204.front.sepia.ceph.com': 'permissive'} 2024-05-03T20:18:06.394 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T20:18:06.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T20:18:06.418 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T20:18:06.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T20:18:06.445 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T20:18:06.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T20:18:06.473 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-03T20:18:06.473 DEBUG:teuthology.orchestra.run.smithi181:> sudo /usr/sbin/setenforce permissive 2024-05-03T20:18:06.497 DEBUG:teuthology.orchestra.run.smithi196:> sudo /usr/sbin/setenforce permissive 2024-05-03T20:18:06.529 DEBUG:teuthology.orchestra.run.smithi204:> sudo /usr/sbin/setenforce permissive 2024-05-03T20:18:06.555 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-03T20:18:06.565 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-03T20:18:06.582 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-03T20:18:06.583 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi181.front.sepia.ceph.com,smithi196.front.sepia.ceph.com,smithi204.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-03T20:22:33.930 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi181.front.sepia.ceph.com'), Remote(name='ubuntu@smithi196.front.sepia.ceph.com'), Remote(name='ubuntu@smithi204.front.sepia.ceph.com')] 2024-05-03T20:22:33.931 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-05-03T20:22:33.932 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T20:22:34.008 DEBUG:teuthology.orchestra.run.smithi181:> true 2024-05-03T20:22:34.085 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-05-03T20:22:34.085 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-05-03T20:22:34.085 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T20:22:34.160 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-05-03T20:22:34.237 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-05-03T20:22:34.237 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-05-03T20:22:34.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T20:22:34.312 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-05-03T20:22:34.385 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-05-03T20:22:34.386 INFO:teuthology.run_tasks:Running task clock... 2024-05-03T20:22:34.397 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-03T20:22:34.397 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-03T20:22:34.397 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T20:22:34.400 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-03T20:22:34.400 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T20:22:34.402 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-03T20:22:34.402 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T20:22:34.426 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-03T20:22:34.429 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-03T20:22:34.440 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-03T20:22:34.443 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-03T20:22:34.455 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-03T20:22:34.462 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: ntpd: command not found 2024-05-03T20:22:34.466 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: ntpd: command not found 2024-05-03T20:22:34.468 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-03T20:22:34.473 INFO:teuthology.orchestra.run.smithi181.stdout:506 Cannot talk to daemon 2024-05-03T20:22:34.476 INFO:teuthology.orchestra.run.smithi196.stdout:506 Cannot talk to daemon 2024-05-03T20:22:34.486 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-03T20:22:34.489 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-03T20:22:34.491 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: ntpd: command not found 2024-05-03T20:22:34.499 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-03T20:22:34.501 INFO:teuthology.orchestra.run.smithi204.stdout:506 Cannot talk to daemon 2024-05-03T20:22:34.502 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-03T20:22:34.515 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-03T20:22:34.528 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-03T20:22:34.549 INFO:teuthology.orchestra.run.smithi181.stderr:bash: line 1: ntpq: command not found 2024-05-03T20:22:34.552 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T20:22:34.552 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2024-05-03T20:22:34.552 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T20:22:34.553 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T20:22:34.553 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T20:22:34.553 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T20:22:34.558 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-05-03T20:22:34.561 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T20:22:34.561 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-05-03T20:22:34.561 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T20:22:34.561 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T20:22:34.561 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T20:22:34.562 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T20:22:34.582 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-05-03T20:22:34.586 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T20:22:34.586 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-05-03T20:22:34.586 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T20:22:34.586 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T20:22:34.586 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T20:22:34.586 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T20:22:34.587 INFO:teuthology.run_tasks:Running task install... 2024-05-03T20:22:34.595 DEBUG:teuthology.task.install:project ceph 2024-05-03T20:22:34.597 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea'}} 2024-05-03T20:22:34.597 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea'} 2024-05-03T20:22:34.597 INFO:teuthology.task.install:Using flavor: default 2024-05-03T20:22:34.603 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-03T20:22:34.603 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-03T20:22:34.604 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea', 'tag': None, 'wait_for_package': False} 2024-05-03T20:22:34.604 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T20:22:34.606 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea', 'tag': None, 'wait_for_package': False} 2024-05-03T20:22:34.606 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T20:22:34.607 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea', 'tag': None, 'wait_for_package': False} 2024-05-03T20:22:34.607 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T20:22:34.757 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/ 2024-05-03T20:22:34.757 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3450.g9a047fe7 2024-05-03T20:22:34.824 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/ 2024-05-03T20:22:34.824 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3450.g9a047fe7 2024-05-03T20:22:34.835 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/ 2024-05-03T20:22:34.835 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3450.g9a047fe7 2024-05-03T20:22:34.868 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-03T20:22:34.868 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-03T20:22:34.868 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-03T20:22:34.892 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-03T20:22:34.893 DEBUG:teuthology.orchestra.run.smithi204:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9a047fe7599a549042d06a3fb5b6813a203eabea/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-03T20:22:34.938 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-03T20:22:34.938 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-03T20:22:34.938 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-03T20:22:34.957 DEBUG:teuthology.orchestra.run.smithi204:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-03T20:22:34.964 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-03T20:22:34.964 DEBUG:teuthology.orchestra.run.smithi181:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9a047fe7599a549042d06a3fb5b6813a203eabea/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-03T20:22:35.004 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-03T20:22:35.004 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-03T20:22:35.004 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-03T20:22:35.029 DEBUG:teuthology.orchestra.run.smithi181:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-03T20:22:35.030 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-03T20:22:35.030 DEBUG:teuthology.orchestra.run.smithi196:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9a047fe7599a549042d06a3fb5b6813a203eabea/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-03T20:22:35.031 DEBUG:teuthology.orchestra.run.smithi204:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-03T20:22:35.094 INFO:teuthology.orchestra.run.smithi204.stdout:check_obsoletes = 1 2024-05-03T20:22:35.143 DEBUG:teuthology.orchestra.run.smithi181:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-03T20:22:35.144 DEBUG:teuthology.orchestra.run.smithi196:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-03T20:22:35.144 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum clean all 2024-05-03T20:22:35.206 INFO:teuthology.orchestra.run.smithi181.stdout:check_obsoletes = 1 2024-05-03T20:22:35.208 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum clean all 2024-05-03T20:22:35.217 DEBUG:teuthology.orchestra.run.smithi196:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-03T20:22:35.280 INFO:teuthology.orchestra.run.smithi196.stdout:check_obsoletes = 1 2024-05-03T20:22:35.281 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum clean all 2024-05-03T20:22:35.371 INFO:teuthology.orchestra.run.smithi204.stdout:50 files removed 2024-05-03T20:22:35.398 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-03T20:22:35.431 INFO:teuthology.orchestra.run.smithi181.stdout:50 files removed 2024-05-03T20:22:35.458 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-03T20:22:35.506 INFO:teuthology.orchestra.run.smithi196.stdout:50 files removed 2024-05-03T20:22:35.534 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-03T20:22:36.035 INFO:teuthology.orchestra.run.smithi204.stdout:ceph packages for x86_64 221 kB/s | 84 kB 00:00 2024-05-03T20:22:36.139 INFO:teuthology.orchestra.run.smithi181.stdout:ceph packages for x86_64 200 kB/s | 84 kB 00:00 2024-05-03T20:22:36.204 INFO:teuthology.orchestra.run.smithi196.stdout:ceph packages for x86_64 205 kB/s | 84 kB 00:00 2024-05-03T20:22:36.468 INFO:teuthology.orchestra.run.smithi204.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-05-03T20:22:36.514 INFO:teuthology.orchestra.run.smithi181.stdout:ceph noarch packages 36 kB/s | 12 kB 00:00 2024-05-03T20:22:36.871 INFO:teuthology.orchestra.run.smithi196.stdout:ceph noarch packages 19 kB/s | 12 kB 00:00 2024-05-03T20:22:36.881 INFO:teuthology.orchestra.run.smithi181.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-05-03T20:22:36.886 INFO:teuthology.orchestra.run.smithi204.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-05-03T20:22:37.297 INFO:teuthology.orchestra.run.smithi196.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-05-03T20:22:37.690 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-05-03T20:22:38.083 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-05-03T20:22:38.586 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.0 MB 00:01 2024-05-03T20:22:39.807 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-05-03T20:22:40.487 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-05-03T20:22:42.116 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - AppStream 5.7 MB/s | 19 MB 00:03 2024-05-03T20:22:44.699 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.2 MB 00:01 2024-05-03T20:22:45.738 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.2 MB 00:01 2024-05-03T20:22:46.116 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 16 kB 00:00 2024-05-03T20:22:47.370 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-05-03T20:22:47.566 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.2 MB 00:01 2024-05-03T20:22:48.072 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-05-03T20:22:49.476 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 16 kB 00:00 2024-05-03T20:22:50.076 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-05-03T20:22:50.854 INFO:teuthology.orchestra.run.smithi181.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-05-03T20:22:52.417 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-05-03T20:22:54.568 INFO:teuthology.orchestra.run.smithi196.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T20:22:54.569 INFO:teuthology.orchestra.run.smithi196.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T20:22:54.580 INFO:teuthology.orchestra.run.smithi196.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-03T20:22:54.580 INFO:teuthology.orchestra.run.smithi196.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-03T20:22:54.581 INFO:teuthology.orchestra.run.smithi196.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-03T20:22:54.582 INFO:teuthology.orchestra.run.smithi196.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-03T20:22:54.584 INFO:teuthology.orchestra.run.smithi196.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-03T20:22:54.586 INFO:teuthology.orchestra.run.smithi196.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-03T20:22:54.587 INFO:teuthology.orchestra.run.smithi196.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-03T20:22:54.649 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-05-03T20:22:54.664 INFO:teuthology.orchestra.run.smithi196.stdout:======================================================================================= 2024-05-03T20:22:54.664 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-05-03T20:22:54.664 INFO:teuthology.orchestra.run.smithi196.stdout:======================================================================================= 2024-05-03T20:22:54.664 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-05-03T20:22:54.665 INFO:teuthology.orchestra.run.smithi196.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-03T20:22:54.665 INFO:teuthology.orchestra.run.smithi196.stdout: ceph x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 6.6 k 2024-05-03T20:22:54.665 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T20:22:54.665 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 891 k 2024-05-03T20:22:54.665 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 145 k 2024-05-03T20:22:54.665 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 1.5 M 2024-05-03T20:22:54.665 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 142 k 2024-05-03T20:22:54.665 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 3.5 M 2024-05-03T20:22:54.665 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 7.4 M 2024-05-03T20:22:54.665 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 49 k 2024-05-03T20:22:54.665 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 11 M 2024-05-03T20:22:54.665 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 49 M 2024-05-03T20:22:54.666 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 273 k 2024-05-03T20:22:54.666 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 763 k 2024-05-03T20:22:54.666 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 225 k 2024-05-03T20:22:54.666 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-shell x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 27 k 2024-05-03T20:22:54.666 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-top noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 18 k 2024-05-03T20:22:54.666 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-03T20:22:54.666 INFO:teuthology.orchestra.run.smithi196.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-03T20:22:54.666 INFO:teuthology.orchestra.run.smithi196.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-03T20:22:54.666 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 32 k 2024-05-03T20:22:54.666 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 747 k 2024-05-03T20:22:54.666 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 127 k 2024-05-03T20:22:54.666 INFO:teuthology.orchestra.run.smithi196.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-03T20:22:54.666 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-03T20:22:54.666 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-03T20:22:54.667 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-03T20:22:54.667 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-03T20:22:54.667 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-03T20:22:54.667 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 161 k 2024-05-03T20:22:54.667 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 320 k 2024-05-03T20:22:54.667 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 298 k 2024-05-03T20:22:54.667 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 100 k 2024-05-03T20:22:54.667 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 85 k 2024-05-03T20:22:54.667 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T20:22:54.667 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 170 k 2024-05-03T20:22:54.667 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-03T20:22:54.667 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrading: 2024-05-03T20:22:54.667 INFO:teuthology.orchestra.run.smithi196.stdout: librados2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.4 M 2024-05-03T20:22:54.668 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T20:22:54.668 INFO:teuthology.orchestra.run.smithi196.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-03T20:22:54.668 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-03T20:22:54.668 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-05-03T20:22:54.668 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-03T20:22:54.668 INFO:teuthology.orchestra.run.smithi196.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-03T20:22:54.668 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-03T20:22:54.668 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 21 M 2024-05-03T20:22:54.668 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 34 k 2024-05-03T20:22:54.668 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 2.4 M 2024-05-03T20:22:54.668 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 274 k 2024-05-03T20:22:54.668 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 4.7 M 2024-05-03T20:22:54.668 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 17 M 2024-05-03T20:22:54.668 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 16 k 2024-05-03T20:22:54.669 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 25 k 2024-05-03T20:22:54.669 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-03T20:22:54.669 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-03T20:22:54.669 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-03T20:22:54.669 INFO:teuthology.orchestra.run.smithi196.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-03T20:22:54.669 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-03T20:22:54.669 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-03T20:22:54.669 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-03T20:22:54.669 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-03T20:22:54.669 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 163 k 2024-05-03T20:22:54.669 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-03T20:22:54.669 INFO:teuthology.orchestra.run.smithi196.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-03T20:22:54.669 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-03T20:22:54.670 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-03T20:22:54.670 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-03T20:22:54.670 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 500 k 2024-05-03T20:22:54.670 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-03T20:22:54.670 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T20:22:54.670 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-03T20:22:54.670 INFO:teuthology.orchestra.run.smithi196.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-03T20:22:54.670 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-03T20:22:54.670 INFO:teuthology.orchestra.run.smithi196.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-03T20:22:54.670 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-03T20:22:54.670 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-03T20:22:54.670 INFO:teuthology.orchestra.run.smithi196.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-03T20:22:54.670 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-03T20:22:54.671 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-03T20:22:54.671 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-03T20:22:54.671 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-03T20:22:54.671 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-03T20:22:54.671 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-03T20:22:54.671 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-03T20:22:54.671 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-03T20:22:54.671 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-03T20:22:54.671 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-03T20:22:54.671 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 45 k 2024-05-03T20:22:54.671 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 135 k 2024-05-03T20:22:54.671 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-03T20:22:54.671 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-03T20:22:54.671 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-03T20:22:54.672 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-03T20:22:54.672 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-03T20:22:54.672 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-03T20:22:54.672 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-03T20:22:54.672 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-03T20:22:54.672 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-03T20:22:54.672 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-03T20:22:54.672 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-03T20:22:54.672 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-03T20:22:54.672 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-03T20:22:54.672 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-03T20:22:54.672 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-03T20:22:54.672 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-03T20:22:54.673 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-03T20:22:54.673 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-03T20:22:54.673 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-03T20:22:54.673 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-03T20:22:54.673 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-03T20:22:54.673 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-03T20:22:54.673 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-03T20:22:54.673 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-03T20:22:54.673 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-03T20:22:54.673 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-03T20:22:54.673 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-03T20:22:54.673 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-03T20:22:54.673 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-03T20:22:54.673 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-03T20:22:54.674 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-03T20:22:54.674 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-03T20:22:54.674 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-03T20:22:54.674 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-03T20:22:54.674 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-03T20:22:54.674 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-03T20:22:54.674 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-03T20:22:54.674 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-03T20:22:54.674 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-03T20:22:54.674 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-03T20:22:54.674 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-03T20:22:54.674 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-03T20:22:54.674 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-03T20:22:54.675 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-03T20:22:54.675 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-03T20:22:54.675 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-03T20:22:54.675 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-03T20:22:54.675 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-03T20:22:54.675 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-03T20:22:54.675 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-03T20:22:54.675 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-03T20:22:54.675 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-03T20:22:54.675 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-03T20:22:54.675 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-03T20:22:54.675 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-03T20:22:54.675 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-03T20:22:54.675 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-03T20:22:54.676 INFO:teuthology.orchestra.run.smithi196.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-03T20:22:54.676 INFO:teuthology.orchestra.run.smithi196.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-03T20:22:54.676 INFO:teuthology.orchestra.run.smithi196.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-03T20:22:54.676 INFO:teuthology.orchestra.run.smithi196.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-03T20:22:54.676 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-03T20:22:54.676 INFO:teuthology.orchestra.run.smithi196.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-03T20:22:54.676 INFO:teuthology.orchestra.run.smithi196.stdout:Installing weak dependencies: 2024-05-03T20:22:54.676 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-03T20:22:54.676 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-03T20:22:54.676 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-05-03T20:22:54.676 INFO:teuthology.orchestra.run.smithi196.stdout:======================================================================================= 2024-05-03T20:22:54.676 INFO:teuthology.orchestra.run.smithi196.stdout:Install 148 Packages 2024-05-03T20:22:54.676 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrade 4 Packages 2024-05-03T20:22:54.676 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-03T20:22:54.677 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 222 M 2024-05-03T20:22:54.677 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-05-03T20:22:55.188 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 9.6 kB/s | 1.7 kB 00:00 2024-05-03T20:22:55.943 INFO:teuthology.orchestra.run.smithi181.stdout:lab-extras 9.6 kB/s | 1.7 kB 00:00 2024-05-03T20:22:56.060 INFO:teuthology.orchestra.run.smithi196.stdout:(1/152): ceph-19.0.0-3450.g9a047fe7.el9.x86_64. 14 kB/s | 6.6 kB 00:00 2024-05-03T20:22:56.518 INFO:teuthology.orchestra.run.smithi196.stdout:(2/152): ceph-base-19.0.0-3450.g9a047fe7.el9.x8 5.9 MB/s | 5.4 MB 00:00 2024-05-03T20:22:56.560 INFO:teuthology.orchestra.run.smithi196.stdout:(3/152): ceph-immutable-object-cache-19.0.0-345 3.4 MB/s | 145 kB 00:00 2024-05-03T20:22:56.819 INFO:teuthology.orchestra.run.smithi196.stdout:(4/152): ceph-mds-19.0.0-3450.g9a047fe7.el9.x86 9.2 MB/s | 2.4 MB 00:00 2024-05-03T20:22:57.002 INFO:teuthology.orchestra.run.smithi196.stdout:(5/152): ceph-mgr-19.0.0-3450.g9a047fe7.el9.x86 8.3 MB/s | 1.5 MB 00:00 2024-05-03T20:22:57.078 INFO:teuthology.orchestra.run.smithi196.stdout:(6/152): ceph-fuse-19.0.0-3450.g9a047fe7.el9.x8 876 kB/s | 891 kB 00:01 2024-05-03T20:22:57.334 INFO:teuthology.orchestra.run.smithi204.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T20:22:57.334 INFO:teuthology.orchestra.run.smithi204.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T20:22:57.345 INFO:teuthology.orchestra.run.smithi204.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-03T20:22:57.346 INFO:teuthology.orchestra.run.smithi204.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-03T20:22:57.347 INFO:teuthology.orchestra.run.smithi204.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-03T20:22:57.348 INFO:teuthology.orchestra.run.smithi204.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-03T20:22:57.349 INFO:teuthology.orchestra.run.smithi204.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-03T20:22:57.352 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-03T20:22:57.353 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-03T20:22:57.361 INFO:teuthology.orchestra.run.smithi196.stdout:(7/152): ceph-common-19.0.0-3450.g9a047fe7.el9. 12 MB/s | 21 MB 00:01 2024-05-03T20:22:57.413 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-05-03T20:22:57.434 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================= 2024-05-03T20:22:57.435 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-05-03T20:22:57.435 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================= 2024-05-03T20:22:57.435 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-05-03T20:22:57.435 INFO:teuthology.orchestra.run.smithi204.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-03T20:22:57.435 INFO:teuthology.orchestra.run.smithi204.stdout: ceph x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 6.6 k 2024-05-03T20:22:57.435 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T20:22:57.435 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 891 k 2024-05-03T20:22:57.435 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 145 k 2024-05-03T20:22:57.435 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 1.5 M 2024-05-03T20:22:57.435 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 142 k 2024-05-03T20:22:57.435 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 3.5 M 2024-05-03T20:22:57.435 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 7.4 M 2024-05-03T20:22:57.435 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 49 k 2024-05-03T20:22:57.435 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 11 M 2024-05-03T20:22:57.436 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 49 M 2024-05-03T20:22:57.436 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 273 k 2024-05-03T20:22:57.436 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 763 k 2024-05-03T20:22:57.436 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 225 k 2024-05-03T20:22:57.436 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 27 k 2024-05-03T20:22:57.436 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 18 k 2024-05-03T20:22:57.436 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-03T20:22:57.436 INFO:teuthology.orchestra.run.smithi204.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-03T20:22:57.436 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-03T20:22:57.436 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 32 k 2024-05-03T20:22:57.436 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 747 k 2024-05-03T20:22:57.436 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 127 k 2024-05-03T20:22:57.436 INFO:teuthology.orchestra.run.smithi204.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-03T20:22:57.436 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-03T20:22:57.436 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-03T20:22:57.437 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-03T20:22:57.437 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-03T20:22:57.437 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-03T20:22:57.437 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 161 k 2024-05-03T20:22:57.437 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 320 k 2024-05-03T20:22:57.437 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 298 k 2024-05-03T20:22:57.437 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 100 k 2024-05-03T20:22:57.437 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 85 k 2024-05-03T20:22:57.437 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T20:22:57.437 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 170 k 2024-05-03T20:22:57.437 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-03T20:22:57.437 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrading: 2024-05-03T20:22:57.437 INFO:teuthology.orchestra.run.smithi204.stdout: librados2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.4 M 2024-05-03T20:22:57.437 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T20:22:57.437 INFO:teuthology.orchestra.run.smithi204.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-03T20:22:57.438 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-03T20:22:57.438 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-05-03T20:22:57.438 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-03T20:22:57.438 INFO:teuthology.orchestra.run.smithi204.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-03T20:22:57.438 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-03T20:22:57.438 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 21 M 2024-05-03T20:22:57.438 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 34 k 2024-05-03T20:22:57.438 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 2.4 M 2024-05-03T20:22:57.438 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 274 k 2024-05-03T20:22:57.438 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 4.7 M 2024-05-03T20:22:57.438 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 17 M 2024-05-03T20:22:57.438 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 16 k 2024-05-03T20:22:57.438 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 25 k 2024-05-03T20:22:57.438 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-03T20:22:57.439 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-03T20:22:57.439 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-03T20:22:57.439 INFO:teuthology.orchestra.run.smithi204.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-03T20:22:57.439 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-03T20:22:57.439 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-03T20:22:57.439 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-03T20:22:57.439 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-03T20:22:57.439 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 163 k 2024-05-03T20:22:57.439 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-03T20:22:57.439 INFO:teuthology.orchestra.run.smithi204.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-03T20:22:57.439 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-03T20:22:57.439 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-03T20:22:57.439 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-03T20:22:57.439 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 500 k 2024-05-03T20:22:57.439 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-03T20:22:57.440 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T20:22:57.440 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-03T20:22:57.440 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-03T20:22:57.440 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-03T20:22:57.440 INFO:teuthology.orchestra.run.smithi204.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-03T20:22:57.440 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-03T20:22:57.440 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-03T20:22:57.440 INFO:teuthology.orchestra.run.smithi204.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-03T20:22:57.440 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-03T20:22:57.440 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-03T20:22:57.440 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-03T20:22:57.440 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-03T20:22:57.440 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-03T20:22:57.440 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-03T20:22:57.440 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-03T20:22:57.441 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-03T20:22:57.441 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-03T20:22:57.441 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-03T20:22:57.441 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 45 k 2024-05-03T20:22:57.441 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 135 k 2024-05-03T20:22:57.441 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-03T20:22:57.441 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-03T20:22:57.441 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-03T20:22:57.441 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-03T20:22:57.441 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-03T20:22:57.441 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-03T20:22:57.441 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-03T20:22:57.441 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-03T20:22:57.441 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-03T20:22:57.441 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-03T20:22:57.442 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-03T20:22:57.442 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-03T20:22:57.442 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-03T20:22:57.442 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-03T20:22:57.442 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-03T20:22:57.442 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-03T20:22:57.442 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-03T20:22:57.442 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-03T20:22:57.442 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-03T20:22:57.442 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-03T20:22:57.442 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-03T20:22:57.442 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-03T20:22:57.442 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-03T20:22:57.443 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-03T20:22:57.443 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-03T20:22:57.443 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-03T20:22:57.443 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-03T20:22:57.443 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-03T20:22:57.443 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-03T20:22:57.443 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-03T20:22:57.443 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-03T20:22:57.443 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-03T20:22:57.443 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-03T20:22:57.443 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-03T20:22:57.443 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-03T20:22:57.443 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-03T20:22:57.443 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-03T20:22:57.443 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-03T20:22:57.444 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-03T20:22:57.444 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-03T20:22:57.444 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-03T20:22:57.444 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-03T20:22:57.444 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-03T20:22:57.444 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-03T20:22:57.444 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-03T20:22:57.444 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-03T20:22:57.444 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-03T20:22:57.444 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-03T20:22:57.444 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-03T20:22:57.444 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-03T20:22:57.444 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-03T20:22:57.444 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-03T20:22:57.444 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-03T20:22:57.445 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-03T20:22:57.445 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-03T20:22:57.445 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-03T20:22:57.445 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-03T20:22:57.445 INFO:teuthology.orchestra.run.smithi204.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-03T20:22:57.445 INFO:teuthology.orchestra.run.smithi204.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-03T20:22:57.445 INFO:teuthology.orchestra.run.smithi204.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-03T20:22:57.445 INFO:teuthology.orchestra.run.smithi204.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-03T20:22:57.445 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-03T20:22:57.445 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-03T20:22:57.445 INFO:teuthology.orchestra.run.smithi204.stdout:Installing weak dependencies: 2024-05-03T20:22:57.445 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-03T20:22:57.445 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-03T20:22:57.445 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-05-03T20:22:57.445 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================= 2024-05-03T20:22:57.445 INFO:teuthology.orchestra.run.smithi204.stdout:Install 148 Packages 2024-05-03T20:22:57.446 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrade 4 Packages 2024-05-03T20:22:57.446 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-03T20:22:57.446 INFO:teuthology.orchestra.run.smithi196.stdout:(8/152): ceph-mon-19.0.0-3450.g9a047fe7.el9.x86 11 MB/s | 4.7 MB 00:00 2024-05-03T20:22:57.447 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 222 M 2024-05-03T20:22:57.447 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-05-03T20:22:57.487 INFO:teuthology.orchestra.run.smithi196.stdout:(9/152): ceph-selinux-19.0.0-3450.g9a047fe7.el9 504 kB/s | 25 kB 00:00 2024-05-03T20:22:58.069 INFO:teuthology.orchestra.run.smithi181.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T20:22:58.069 INFO:teuthology.orchestra.run.smithi181.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T20:22:58.080 INFO:teuthology.orchestra.run.smithi181.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-03T20:22:58.081 INFO:teuthology.orchestra.run.smithi181.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-03T20:22:58.081 INFO:teuthology.orchestra.run.smithi181.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-03T20:22:58.082 INFO:teuthology.orchestra.run.smithi181.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-03T20:22:58.084 INFO:teuthology.orchestra.run.smithi181.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-03T20:22:58.086 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-03T20:22:58.087 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-03T20:22:58.092 INFO:teuthology.orchestra.run.smithi204.stdout:(1/152): ceph-19.0.0-3450.g9a047fe7.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-05-03T20:22:58.146 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-05-03T20:22:58.155 INFO:teuthology.orchestra.run.smithi181.stdout:======================================================================================= 2024-05-03T20:22:58.155 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2024-05-03T20:22:58.155 INFO:teuthology.orchestra.run.smithi181.stdout:======================================================================================= 2024-05-03T20:22:58.155 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2024-05-03T20:22:58.155 INFO:teuthology.orchestra.run.smithi181.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-03T20:22:58.155 INFO:teuthology.orchestra.run.smithi181.stdout: ceph x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 6.6 k 2024-05-03T20:22:58.155 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T20:22:58.155 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 891 k 2024-05-03T20:22:58.155 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 145 k 2024-05-03T20:22:58.156 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 1.5 M 2024-05-03T20:22:58.156 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 142 k 2024-05-03T20:22:58.156 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 3.5 M 2024-05-03T20:22:58.156 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 7.4 M 2024-05-03T20:22:58.156 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 49 k 2024-05-03T20:22:58.156 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 11 M 2024-05-03T20:22:58.156 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 49 M 2024-05-03T20:22:58.156 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 273 k 2024-05-03T20:22:58.156 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 763 k 2024-05-03T20:22:58.156 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 225 k 2024-05-03T20:22:58.156 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-shell x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 27 k 2024-05-03T20:22:58.156 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 18 k 2024-05-03T20:22:58.156 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-03T20:22:58.156 INFO:teuthology.orchestra.run.smithi181.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-03T20:22:58.156 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-03T20:22:58.157 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 32 k 2024-05-03T20:22:58.157 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 747 k 2024-05-03T20:22:58.157 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 127 k 2024-05-03T20:22:58.157 INFO:teuthology.orchestra.run.smithi181.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-03T20:22:58.157 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-03T20:22:58.157 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-03T20:22:58.157 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-03T20:22:58.157 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-03T20:22:58.157 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-03T20:22:58.157 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 161 k 2024-05-03T20:22:58.157 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 320 k 2024-05-03T20:22:58.157 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 298 k 2024-05-03T20:22:58.157 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 100 k 2024-05-03T20:22:58.157 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 85 k 2024-05-03T20:22:58.158 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T20:22:58.158 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 170 k 2024-05-03T20:22:58.158 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-03T20:22:58.158 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrading: 2024-05-03T20:22:58.158 INFO:teuthology.orchestra.run.smithi181.stdout: librados2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.4 M 2024-05-03T20:22:58.158 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T20:22:58.158 INFO:teuthology.orchestra.run.smithi181.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-03T20:22:58.158 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-03T20:22:58.158 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2024-05-03T20:22:58.158 INFO:teuthology.orchestra.run.smithi181.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-03T20:22:58.158 INFO:teuthology.orchestra.run.smithi181.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-03T20:22:58.158 INFO:teuthology.orchestra.run.smithi181.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-03T20:22:58.158 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 21 M 2024-05-03T20:22:58.159 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 34 k 2024-05-03T20:22:58.159 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 2.4 M 2024-05-03T20:22:58.159 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 274 k 2024-05-03T20:22:58.159 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 4.7 M 2024-05-03T20:22:58.159 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 17 M 2024-05-03T20:22:58.159 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 16 k 2024-05-03T20:22:58.159 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 25 k 2024-05-03T20:22:58.159 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-03T20:22:58.159 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-03T20:22:58.159 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-03T20:22:58.159 INFO:teuthology.orchestra.run.smithi181.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-03T20:22:58.160 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-03T20:22:58.160 INFO:teuthology.orchestra.run.smithi181.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-03T20:22:58.160 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-03T20:22:58.160 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-03T20:22:58.160 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 163 k 2024-05-03T20:22:58.160 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-03T20:22:58.160 INFO:teuthology.orchestra.run.smithi181.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-03T20:22:58.160 INFO:teuthology.orchestra.run.smithi181.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-03T20:22:58.160 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-03T20:22:58.160 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-03T20:22:58.160 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 500 k 2024-05-03T20:22:58.160 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-03T20:22:58.161 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T20:22:58.161 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-03T20:22:58.161 INFO:teuthology.orchestra.run.smithi181.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-03T20:22:58.161 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-03T20:22:58.161 INFO:teuthology.orchestra.run.smithi181.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-03T20:22:58.161 INFO:teuthology.orchestra.run.smithi181.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-03T20:22:58.161 INFO:teuthology.orchestra.run.smithi181.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-03T20:22:58.161 INFO:teuthology.orchestra.run.smithi181.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-03T20:22:58.161 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-03T20:22:58.161 INFO:teuthology.orchestra.run.smithi181.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-03T20:22:58.161 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-03T20:22:58.161 INFO:teuthology.orchestra.run.smithi181.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-03T20:22:58.162 INFO:teuthology.orchestra.run.smithi181.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-03T20:22:58.162 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-03T20:22:58.162 INFO:teuthology.orchestra.run.smithi181.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-03T20:22:58.162 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-03T20:22:58.162 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-03T20:22:58.162 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-03T20:22:58.162 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 45 k 2024-05-03T20:22:58.162 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 135 k 2024-05-03T20:22:58.162 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-03T20:22:58.162 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-03T20:22:58.162 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-03T20:22:58.163 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-03T20:22:58.163 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-03T20:22:58.163 INFO:teuthology.orchestra.run.smithi181.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-03T20:22:58.163 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-03T20:22:58.163 INFO:teuthology.orchestra.run.smithi181.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-03T20:22:58.163 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-03T20:22:58.163 INFO:teuthology.orchestra.run.smithi181.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-03T20:22:58.163 INFO:teuthology.orchestra.run.smithi181.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-03T20:22:58.163 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-03T20:22:58.163 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-03T20:22:58.163 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-03T20:22:58.163 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-03T20:22:58.163 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-03T20:22:58.164 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-03T20:22:58.164 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-03T20:22:58.164 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-03T20:22:58.164 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-03T20:22:58.164 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-03T20:22:58.164 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-03T20:22:58.164 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-03T20:22:58.164 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-03T20:22:58.164 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-03T20:22:58.164 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-03T20:22:58.164 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-03T20:22:58.164 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-03T20:22:58.164 INFO:teuthology.orchestra.run.smithi181.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-03T20:22:58.164 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-03T20:22:58.165 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-03T20:22:58.165 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-03T20:22:58.165 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-03T20:22:58.165 INFO:teuthology.orchestra.run.smithi181.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-03T20:22:58.165 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-03T20:22:58.165 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-03T20:22:58.165 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-03T20:22:58.165 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-03T20:22:58.165 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-03T20:22:58.165 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-03T20:22:58.165 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-03T20:22:58.165 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-03T20:22:58.165 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-03T20:22:58.165 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-03T20:22:58.165 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-03T20:22:58.166 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-03T20:22:58.166 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-03T20:22:58.166 INFO:teuthology.orchestra.run.smithi181.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-03T20:22:58.166 INFO:teuthology.orchestra.run.smithi181.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-03T20:22:58.166 INFO:teuthology.orchestra.run.smithi181.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-03T20:22:58.166 INFO:teuthology.orchestra.run.smithi181.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-03T20:22:58.166 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-03T20:22:58.166 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-03T20:22:58.166 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-03T20:22:58.166 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-03T20:22:58.166 INFO:teuthology.orchestra.run.smithi181.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-03T20:22:58.166 INFO:teuthology.orchestra.run.smithi181.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-03T20:22:58.166 INFO:teuthology.orchestra.run.smithi181.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-03T20:22:58.167 INFO:teuthology.orchestra.run.smithi181.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-03T20:22:58.167 INFO:teuthology.orchestra.run.smithi181.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-03T20:22:58.167 INFO:teuthology.orchestra.run.smithi181.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-03T20:22:58.167 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-03T20:22:58.167 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-03T20:22:58.167 INFO:teuthology.orchestra.run.smithi181.stdout:Installing weak dependencies: 2024-05-03T20:22:58.167 INFO:teuthology.orchestra.run.smithi181.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-03T20:22:58.167 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:22:58.167 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-05-03T20:22:58.167 INFO:teuthology.orchestra.run.smithi181.stdout:======================================================================================= 2024-05-03T20:22:58.167 INFO:teuthology.orchestra.run.smithi181.stdout:Install 148 Packages 2024-05-03T20:22:58.167 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrade 4 Packages 2024-05-03T20:22:58.167 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:22:58.168 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 222 M 2024-05-03T20:22:58.168 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-05-03T20:22:58.270 INFO:teuthology.orchestra.run.smithi196.stdout:(10/152): ceph-radosgw-19.0.0-3450.g9a047fe7.el 12 MB/s | 11 MB 00:00 2024-05-03T20:22:58.370 INFO:teuthology.orchestra.run.smithi196.stdout:(11/152): cephfs-mirror-19.0.0-3450.g9a047fe7.e 2.2 MB/s | 225 kB 00:00 2024-05-03T20:22:58.462 INFO:teuthology.orchestra.run.smithi196.stdout:(12/152): cephfs-shell-19.0.0-3450.g9a047fe7.el 295 kB/s | 27 kB 00:00 2024-05-03T20:22:58.563 INFO:teuthology.orchestra.run.smithi196.stdout:(13/152): libcephfs-devel-19.0.0-3450.g9a047fe7 317 kB/s | 32 kB 00:00 2024-05-03T20:22:58.600 INFO:teuthology.orchestra.run.smithi204.stdout:(2/152): ceph-fuse-19.0.0-3450.g9a047fe7.el9.x8 1.7 MB/s | 891 kB 00:00 2024-05-03T20:22:59.134 INFO:teuthology.orchestra.run.smithi204.stdout:(3/152): ceph-base-19.0.0-3450.g9a047fe7.el9.x8 3.8 MB/s | 5.4 MB 00:01 2024-05-03T20:22:59.160 INFO:teuthology.orchestra.run.smithi204.stdout:(4/152): ceph-immutable-object-cache-19.0.0-345 259 kB/s | 145 kB 00:00 2024-05-03T20:22:59.271 INFO:teuthology.orchestra.run.smithi196.stdout:(14/152): libcephfs2-19.0.0-3450.g9a047fe7.el9. 1.0 MB/s | 747 kB 00:00 2024-05-03T20:22:59.363 INFO:teuthology.orchestra.run.smithi196.stdout:(15/152): libcephsqlite-19.0.0-3450.g9a047fe7.e 1.7 MB/s | 163 kB 00:00 2024-05-03T20:22:59.445 INFO:teuthology.orchestra.run.smithi181.stdout:(1/152): ceph-19.0.0-3450.g9a047fe7.el9.x86_64. 12 kB/s | 6.6 kB 00:00 2024-05-03T20:22:59.455 INFO:teuthology.orchestra.run.smithi196.stdout:(16/152): librados-devel-19.0.0-3450.g9a047fe7. 1.4 MB/s | 127 kB 00:00 2024-05-03T20:22:59.551 INFO:teuthology.orchestra.run.smithi204.stdout:(5/152): ceph-mds-19.0.0-3450.g9a047fe7.el9.x86 5.7 MB/s | 2.4 MB 00:00 2024-05-03T20:22:59.555 INFO:teuthology.orchestra.run.smithi196.stdout:(17/152): libradosstriper1-19.0.0-3450.g9a047fe 4.9 MB/s | 500 kB 00:00 2024-05-03T20:22:59.918 INFO:teuthology.orchestra.run.smithi204.stdout:(6/152): ceph-mgr-19.0.0-3450.g9a047fe7.el9.x86 2.0 MB/s | 1.5 MB 00:00 2024-05-03T20:23:00.464 INFO:teuthology.orchestra.run.smithi196.stdout:(18/152): librgw2-19.0.0-3450.g9a047fe7.el9.x86 5.9 MB/s | 5.4 MB 00:00 2024-05-03T20:23:00.548 INFO:teuthology.orchestra.run.smithi196.stdout:(19/152): python3-ceph-argparse-19.0.0-3450.g9a 544 kB/s | 45 kB 00:00 2024-05-03T20:23:00.614 INFO:teuthology.orchestra.run.smithi196.stdout:(20/152): python3-ceph-common-19.0.0-3450.g9a04 2.0 MB/s | 135 kB 00:00 2024-05-03T20:23:00.618 INFO:teuthology.orchestra.run.smithi204.stdout:(7/152): ceph-mon-19.0.0-3450.g9a047fe7.el9.x86 4.4 MB/s | 4.7 MB 00:01 2024-05-03T20:23:00.681 INFO:teuthology.orchestra.run.smithi196.stdout:(21/152): python3-cephfs-19.0.0-3450.g9a047fe7. 2.4 MB/s | 161 kB 00:00 2024-05-03T20:23:00.782 INFO:teuthology.orchestra.run.smithi196.stdout:(22/152): python3-rados-19.0.0-3450.g9a047fe7.e 3.1 MB/s | 320 kB 00:00 2024-05-03T20:23:00.882 INFO:teuthology.orchestra.run.smithi196.stdout:(23/152): python3-rbd-19.0.0-3450.g9a047fe7.el9 2.9 MB/s | 298 kB 00:00 2024-05-03T20:23:00.966 INFO:teuthology.orchestra.run.smithi196.stdout:(24/152): python3-rgw-19.0.0-3450.g9a047fe7.el9 1.2 MB/s | 100 kB 00:00 2024-05-03T20:23:01.058 INFO:teuthology.orchestra.run.smithi196.stdout:(25/152): rbd-fuse-19.0.0-3450.g9a047fe7.el9.x8 931 kB/s | 85 kB 00:00 2024-05-03T20:23:01.095 INFO:teuthology.orchestra.run.smithi181.stdout:(2/152): ceph-fuse-19.0.0-3450.g9a047fe7.el9.x8 540 kB/s | 891 kB 00:01 2024-05-03T20:23:01.545 INFO:teuthology.orchestra.run.smithi181.stdout:(3/152): ceph-immutable-object-cache-19.0.0-345 321 kB/s | 145 kB 00:00 2024-05-03T20:23:01.612 INFO:teuthology.orchestra.run.smithi181.stdout:(4/152): ceph-base-19.0.0-3450.g9a047fe7.el9.x8 2.0 MB/s | 5.4 MB 00:02 2024-05-03T20:23:01.820 INFO:teuthology.orchestra.run.smithi181.stdout:(5/152): ceph-mgr-19.0.0-3450.g9a047fe7.el9.x86 7.3 MB/s | 1.5 MB 00:00 2024-05-03T20:23:01.975 INFO:teuthology.orchestra.run.smithi196.stdout:(26/152): ceph-test-19.0.0-3450.g9a047fe7.el9.x 11 MB/s | 49 MB 00:04 2024-05-03T20:23:02.033 INFO:teuthology.orchestra.run.smithi196.stdout:(27/152): rbd-mirror-19.0.0-3450.g9a047fe7.el9. 3.2 MB/s | 3.1 MB 00:00 2024-05-03T20:23:02.058 INFO:teuthology.orchestra.run.smithi196.stdout:(28/152): rbd-nbd-19.0.0-3450.g9a047fe7.el9.x86 2.0 MB/s | 170 kB 00:00 2024-05-03T20:23:02.109 INFO:teuthology.orchestra.run.smithi196.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3450.g 454 kB/s | 34 kB 00:00 2024-05-03T20:23:02.134 INFO:teuthology.orchestra.run.smithi196.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3450.g9a047fe 1.9 MB/s | 142 kB 00:00 2024-05-03T20:23:02.396 INFO:teuthology.orchestra.run.smithi181.stdout:(6/152): ceph-mon-19.0.0-3450.g9a047fe7.el9.x86 8.2 MB/s | 4.7 MB 00:00 2024-05-03T20:23:02.902 INFO:teuthology.orchestra.run.smithi204.stdout:(8/152): ceph-radosgw-19.0.0-3450.g9a047fe7.el9 4.6 MB/s | 11 MB 00:02 2024-05-03T20:23:02.986 INFO:teuthology.orchestra.run.smithi204.stdout:(9/152): ceph-selinux-19.0.0-3450.g9a047fe7.el9 303 kB/s | 25 kB 00:00 2024-05-03T20:23:03.043 INFO:teuthology.orchestra.run.smithi196.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3450.g9a047 3.8 MB/s | 3.5 MB 00:00 2024-05-03T20:23:03.201 INFO:teuthology.orchestra.run.smithi196.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3450.g9a 1.7 MB/s | 274 kB 00:00 2024-05-03T20:23:03.285 INFO:teuthology.orchestra.run.smithi196.stdout:(33/152): ceph-mgr-diskprediction-local-19.0.0- 6.4 MB/s | 7.4 MB 00:01 2024-05-03T20:23:03.310 INFO:teuthology.orchestra.run.smithi196.stdout:(34/152): ceph-mgr-rook-19.0.0-3450.g9a047fe7.e 454 kB/s | 49 kB 00:00 2024-05-03T20:23:03.352 INFO:teuthology.orchestra.run.smithi196.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3450.g9 247 kB/s | 16 kB 00:00 2024-05-03T20:23:03.394 INFO:teuthology.orchestra.run.smithi196.stdout:(36/152): ceph-volume-19.0.0-3450.g9a047fe7.el9 3.2 MB/s | 273 kB 00:00 2024-05-03T20:23:03.478 INFO:teuthology.orchestra.run.smithi196.stdout:(37/152): cephfs-top-19.0.0-3450.g9a047fe7.el9. 219 kB/s | 18 kB 00:00 2024-05-03T20:23:03.511 INFO:teuthology.orchestra.run.smithi196.stdout:(38/152): cephadm-19.0.0-3450.g9a047fe7.el9.noa 4.7 MB/s | 763 kB 00:00 2024-05-03T20:23:03.553 INFO:teuthology.orchestra.run.smithi196.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 80 kB 00:00 2024-05-03T20:23:03.620 INFO:teuthology.orchestra.run.smithi196.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 7.2 MB/s | 803 kB 00:00 2024-05-03T20:23:03.645 INFO:teuthology.orchestra.run.smithi196.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-05-03T20:23:03.712 INFO:teuthology.orchestra.run.smithi196.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-05-03T20:23:03.771 INFO:teuthology.orchestra.run.smithi196.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 9.8 MB/s | 1.2 MB 00:00 2024-05-03T20:23:03.804 INFO:teuthology.orchestra.run.smithi196.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-05-03T20:23:03.846 INFO:teuthology.orchestra.run.smithi196.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-05-03T20:23:03.888 INFO:teuthology.orchestra.run.smithi196.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-05-03T20:23:03.939 INFO:teuthology.orchestra.run.smithi196.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-05-03T20:23:04.155 INFO:teuthology.orchestra.run.smithi196.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 689 kB 00:00 2024-05-03T20:23:04.221 INFO:teuthology.orchestra.run.smithi181.stdout:(7/152): ceph-osd-19.0.0-3450.g9a047fe7.el9.x86 9.2 MB/s | 17 MB 00:01 2024-05-03T20:23:04.231 INFO:teuthology.orchestra.run.smithi196.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-05-03T20:23:04.256 INFO:teuthology.orchestra.run.smithi196.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-05-03T20:23:04.306 INFO:teuthology.orchestra.run.smithi196.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 285 kB/s | 21 kB 00:00 2024-05-03T20:23:04.331 INFO:teuthology.orchestra.run.smithi196.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-05-03T20:23:04.374 INFO:teuthology.orchestra.run.smithi196.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 490 kB/s | 33 kB 00:00 2024-05-03T20:23:04.404 INFO:teuthology.orchestra.run.smithi181.stdout:(8/152): ceph-mds-19.0.0-3450.g9a047fe7.el9.x86 847 kB/s | 2.4 MB 00:02 2024-05-03T20:23:04.463 INFO:teuthology.orchestra.run.smithi181.stdout:(9/152): ceph-selinux-19.0.0-3450.g9a047fe7.el9 433 kB/s | 25 kB 00:00 2024-05-03T20:23:04.465 INFO:teuthology.orchestra.run.smithi196.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-05-03T20:23:04.491 INFO:teuthology.orchestra.run.smithi196.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 156 kB/s | 18 kB 00:00 2024-05-03T20:23:04.533 INFO:teuthology.orchestra.run.smithi196.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 166 kB/s | 11 kB 00:00 2024-05-03T20:23:04.558 INFO:teuthology.orchestra.run.smithi196.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-05-03T20:23:04.718 INFO:teuthology.orchestra.run.smithi196.stdout:(58/152): ceph-osd-19.0.0-3450.g9a047fe7.el9.x8 2.2 MB/s | 17 MB 00:07 2024-05-03T20:23:04.742 INFO:teuthology.orchestra.run.smithi196.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 217 kB/s | 45 kB 00:00 2024-05-03T20:23:04.776 INFO:teuthology.orchestra.run.smithi196.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.0 MB/s | 662 kB 00:00 2024-05-03T20:23:04.809 INFO:teuthology.orchestra.run.smithi196.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 582 kB 00:00 2024-05-03T20:23:04.835 INFO:teuthology.orchestra.run.smithi196.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 276 kB/s | 25 kB 00:00 2024-05-03T20:23:04.860 INFO:teuthology.orchestra.run.smithi196.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 566 kB/s | 47 kB 00:00 2024-05-03T20:23:04.885 INFO:teuthology.orchestra.run.smithi196.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-05-03T20:23:04.910 INFO:teuthology.orchestra.run.smithi196.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-05-03T20:23:04.936 INFO:teuthology.orchestra.run.smithi196.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 468 kB/s | 35 kB 00:00 2024-05-03T20:23:05.094 INFO:teuthology.orchestra.run.smithi196.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 23 MB/s | 4.7 MB 00:00 2024-05-03T20:23:05.161 INFO:teuthology.orchestra.run.smithi196.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-05-03T20:23:05.220 INFO:teuthology.orchestra.run.smithi196.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 7.2 MB/s | 911 kB 00:00 2024-05-03T20:23:05.245 INFO:teuthology.orchestra.run.smithi196.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-05-03T20:23:05.312 INFO:teuthology.orchestra.run.smithi196.stdout:(71/152): postgresql-13.14-1.el9.x86_64.rpm 4.2 MB/s | 1.6 MB 00:00 2024-05-03T20:23:05.354 INFO:teuthology.orchestra.run.smithi196.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-05-03T20:23:05.388 INFO:teuthology.orchestra.run.smithi181.stdout:(10/152): ceph-radosgw-19.0.0-3450.g9a047fe7.el 9.1 MB/s | 11 MB 00:01 2024-05-03T20:23:05.396 INFO:teuthology.orchestra.run.smithi196.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-05-03T20:23:05.438 INFO:teuthology.orchestra.run.smithi181.stdout:(11/152): cephfs-mirror-19.0.0-3450.g9a047fe7.e 4.4 MB/s | 225 kB 00:00 2024-05-03T20:23:05.461 INFO:teuthology.orchestra.run.smithi204.stdout:(10/152): ceph-osd-19.0.0-3450.g9a047fe7.el9.x8 3.0 MB/s | 17 MB 00:05 2024-05-03T20:23:05.479 INFO:teuthology.orchestra.run.smithi196.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-05-03T20:23:05.480 INFO:teuthology.orchestra.run.smithi181.stdout:(12/152): cephfs-shell-19.0.0-3450.g9a047fe7.el 649 kB/s | 27 kB 00:00 2024-05-03T20:23:05.522 INFO:teuthology.orchestra.run.smithi181.stdout:(13/152): libcephfs-devel-19.0.0-3450.g9a047fe7 762 kB/s | 32 kB 00:00 2024-05-03T20:23:05.530 INFO:teuthology.orchestra.run.smithi196.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.8 MB/s | 244 kB 00:00 2024-05-03T20:23:05.555 INFO:teuthology.orchestra.run.smithi196.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-05-03T20:23:05.606 INFO:teuthology.orchestra.run.smithi181.stdout:(14/152): libcephfs2-19.0.0-3450.g9a047fe7.el9. 8.8 MB/s | 747 kB 00:00 2024-05-03T20:23:05.639 INFO:teuthology.orchestra.run.smithi196.stdout:(77/152): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 6.0 MB 00:00 2024-05-03T20:23:05.657 INFO:teuthology.orchestra.run.smithi181.stdout:(15/152): libcephsqlite-19.0.0-3450.g9a047fe7.e 3.2 MB/s | 163 kB 00:00 2024-05-03T20:23:05.664 INFO:teuthology.orchestra.run.smithi196.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 192 kB/s | 26 kB 00:00 2024-05-03T20:23:05.677 INFO:teuthology.orchestra.run.smithi204.stdout:(11/152): ceph-common-19.0.0-3450.g9a047fe7.el9 2.7 MB/s | 21 MB 00:07 2024-05-03T20:23:05.689 INFO:teuthology.orchestra.run.smithi196.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 143 kB/s | 19 kB 00:00 2024-05-03T20:23:05.703 INFO:teuthology.orchestra.run.smithi204.stdout:(12/152): cephfs-mirror-19.0.0-3450.g9a047fe7.e 930 kB/s | 225 kB 00:00 2024-05-03T20:23:05.707 INFO:teuthology.orchestra.run.smithi181.stdout:(16/152): librados-devel-19.0.0-3450.g9a047fe7. 2.5 MB/s | 127 kB 00:00 2024-05-03T20:23:05.714 INFO:teuthology.orchestra.run.smithi196.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-05-03T20:23:05.720 INFO:teuthology.orchestra.run.smithi204.stdout:(13/152): cephfs-shell-19.0.0-3450.g9a047fe7.el 650 kB/s | 27 kB 00:00 2024-05-03T20:23:05.737 INFO:teuthology.orchestra.run.smithi204.stdout:(14/152): libcephfs-devel-19.0.0-3450.g9a047fe7 952 kB/s | 32 kB 00:00 2024-05-03T20:23:05.739 INFO:teuthology.orchestra.run.smithi196.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 461 kB/s | 35 kB 00:00 2024-05-03T20:23:05.774 INFO:teuthology.orchestra.run.smithi181.stdout:(17/152): libradosstriper1-19.0.0-3450.g9a047fe 7.3 MB/s | 500 kB 00:00 2024-05-03T20:23:05.798 INFO:teuthology.orchestra.run.smithi196.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-05-03T20:23:05.804 INFO:teuthology.orchestra.run.smithi204.stdout:(15/152): libcephsqlite-19.0.0-3450.g9a047fe7.e 2.4 MB/s | 163 kB 00:00 2024-05-03T20:23:05.823 INFO:teuthology.orchestra.run.smithi196.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-05-03T20:23:05.871 INFO:teuthology.orchestra.run.smithi204.stdout:(16/152): librados-devel-19.0.0-3450.g9a047fe7. 1.9 MB/s | 127 kB 00:00 2024-05-03T20:23:05.899 INFO:teuthology.orchestra.run.smithi196.stdout:(84/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 5.1 MB 00:00 2024-05-03T20:23:05.922 INFO:teuthology.orchestra.run.smithi204.stdout:(17/152): libcephfs2-19.0.0-3450.g9a047fe7.el9. 3.6 MB/s | 747 kB 00:00 2024-05-03T20:23:05.924 INFO:teuthology.orchestra.run.smithi196.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 615 kB/s | 77 kB 00:00 2024-05-03T20:23:05.949 INFO:teuthology.orchestra.run.smithi196.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 337 kB/s | 42 kB 00:00 2024-05-03T20:23:05.975 INFO:teuthology.orchestra.run.smithi196.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-05-03T20:23:06.000 INFO:teuthology.orchestra.run.smithi196.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-05-03T20:23:06.025 INFO:teuthology.orchestra.run.smithi196.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-05-03T20:23:06.038 INFO:teuthology.orchestra.run.smithi204.stdout:(18/152): libradosstriper1-19.0.0-3450.g9a047fe 2.9 MB/s | 500 kB 00:00 2024-05-03T20:23:06.050 INFO:teuthology.orchestra.run.smithi196.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-05-03T20:23:06.084 INFO:teuthology.orchestra.run.smithi196.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 638 kB/s | 54 kB 00:00 2024-05-03T20:23:06.113 INFO:teuthology.orchestra.run.smithi204.stdout:(19/152): python3-ceph-argparse-19.0.0-3450.g9a 603 kB/s | 45 kB 00:00 2024-05-03T20:23:06.145 INFO:teuthology.orchestra.run.smithi196.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 452 kB/s | 42 kB 00:00 2024-05-03T20:23:06.185 INFO:teuthology.orchestra.run.smithi196.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 440 kB/s | 44 kB 00:00 2024-05-03T20:23:06.205 INFO:teuthology.orchestra.run.smithi204.stdout:(20/152): python3-ceph-common-19.0.0-3450.g9a04 1.4 MB/s | 135 kB 00:00 2024-05-03T20:23:06.216 INFO:teuthology.orchestra.run.smithi181.stdout:(18/152): librgw2-19.0.0-3450.g9a047fe7.el9.x86 12 MB/s | 5.4 MB 00:00 2024-05-03T20:23:06.247 INFO:teuthology.orchestra.run.smithi204.stdout:(21/152): python3-cephfs-19.0.0-3450.g9a047fe7. 3.8 MB/s | 161 kB 00:00 2024-05-03T20:23:06.251 INFO:teuthology.orchestra.run.smithi196.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-05-03T20:23:06.274 INFO:teuthology.orchestra.run.smithi181.stdout:(19/152): python3-ceph-argparse-19.0.0-3450.g9a 773 kB/s | 45 kB 00:00 2024-05-03T20:23:06.285 INFO:teuthology.orchestra.run.smithi196.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 358 kB/s | 36 kB 00:00 2024-05-03T20:23:06.328 INFO:teuthology.orchestra.run.smithi196.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 628 kB/s | 47 kB 00:00 2024-05-03T20:23:06.333 INFO:teuthology.orchestra.run.smithi181.stdout:(20/152): python3-ceph-common-19.0.0-3450.g9a04 2.3 MB/s | 135 kB 00:00 2024-05-03T20:23:06.339 INFO:teuthology.orchestra.run.smithi204.stdout:(22/152): python3-rados-19.0.0-3450.g9a047fe7.e 3.4 MB/s | 320 kB 00:00 2024-05-03T20:23:06.352 INFO:teuthology.orchestra.run.smithi196.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-05-03T20:23:06.383 INFO:teuthology.orchestra.run.smithi181.stdout:(21/152): python3-cephfs-19.0.0-3450.g9a047fe7. 3.2 MB/s | 161 kB 00:00 2024-05-03T20:23:06.423 INFO:teuthology.orchestra.run.smithi204.stdout:(23/152): python3-rbd-19.0.0-3450.g9a047fe7.el9 3.5 MB/s | 298 kB 00:00 2024-05-03T20:23:06.450 INFO:teuthology.orchestra.run.smithi181.stdout:(22/152): python3-rados-19.0.0-3450.g9a047fe7.e 4.7 MB/s | 320 kB 00:00 2024-05-03T20:23:06.473 INFO:teuthology.orchestra.run.smithi204.stdout:(24/152): python3-rgw-19.0.0-3450.g9a047fe7.el9 2.0 MB/s | 100 kB 00:00 2024-05-03T20:23:06.501 INFO:teuthology.orchestra.run.smithi181.stdout:(23/152): python3-rbd-19.0.0-3450.g9a047fe7.el9 5.8 MB/s | 298 kB 00:00 2024-05-03T20:23:06.511 INFO:teuthology.orchestra.run.smithi196.stdout:(98/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-05-03T20:23:06.523 INFO:teuthology.orchestra.run.smithi204.stdout:(25/152): rbd-fuse-19.0.0-3450.g9a047fe7.el9.x8 1.7 MB/s | 85 kB 00:00 2024-05-03T20:23:06.536 INFO:teuthology.orchestra.run.smithi196.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 277 kB/s | 58 kB 00:00 2024-05-03T20:23:06.542 INFO:teuthology.orchestra.run.smithi181.stdout:(24/152): python3-rgw-19.0.0-3450.g9a047fe7.el9 2.3 MB/s | 100 kB 00:00 2024-05-03T20:23:06.553 INFO:teuthology.orchestra.run.smithi196.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 467 kB/s | 94 kB 00:00 2024-05-03T20:23:06.578 INFO:teuthology.orchestra.run.smithi196.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 952 kB/s | 64 kB 00:00 2024-05-03T20:23:06.593 INFO:teuthology.orchestra.run.smithi181.stdout:(25/152): rbd-fuse-19.0.0-3450.g9a047fe7.el9.x8 1.7 MB/s | 85 kB 00:00 2024-05-03T20:23:06.604 INFO:teuthology.orchestra.run.smithi196.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 668 kB/s | 45 kB 00:00 2024-05-03T20:23:06.729 INFO:teuthology.orchestra.run.smithi196.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 127 kB/s | 22 kB 00:00 2024-05-03T20:23:06.821 INFO:teuthology.orchestra.run.smithi196.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 3.5 MB/s | 863 kB 00:00 2024-05-03T20:23:06.885 INFO:teuthology.orchestra.run.smithi181.stdout:(26/152): rbd-mirror-19.0.0-3450.g9a047fe7.el9. 11 MB/s | 3.1 MB 00:00 2024-05-03T20:23:06.952 INFO:teuthology.orchestra.run.smithi181.stdout:(27/152): rbd-nbd-19.0.0-3450.g9a047fe7.el9.x86 2.5 MB/s | 170 kB 00:00 2024-05-03T20:23:06.994 INFO:teuthology.orchestra.run.smithi181.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3450.g 820 kB/s | 34 kB 00:00 2024-05-03T20:23:07.024 INFO:teuthology.orchestra.run.smithi204.stdout:(26/152): librgw2-19.0.0-3450.g9a047fe7.el9.x86 4.9 MB/s | 5.4 MB 00:01 2024-05-03T20:23:07.044 INFO:teuthology.orchestra.run.smithi181.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3450.g9a047fe 2.8 MB/s | 142 kB 00:00 2024-05-03T20:23:07.063 INFO:teuthology.orchestra.run.smithi196.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 80 kB/s | 19 kB 00:00 2024-05-03T20:23:07.074 INFO:teuthology.orchestra.run.smithi204.stdout:(27/152): rbd-nbd-19.0.0-3450.g9a047fe7.el9.x86 3.3 MB/s | 170 kB 00:00 2024-05-03T20:23:07.141 INFO:teuthology.orchestra.run.smithi204.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3450.g 514 kB/s | 34 kB 00:00 2024-05-03T20:23:07.172 INFO:teuthology.orchestra.run.smithi196.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 966 kB/s | 548 kB 00:00 2024-05-03T20:23:07.222 INFO:teuthology.orchestra.run.smithi196.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 627 kB/s | 309 kB 00:00 2024-05-03T20:23:07.266 INFO:teuthology.orchestra.run.smithi204.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3450.g9a047fe 1.1 MB/s | 142 kB 00:00 2024-05-03T20:23:07.325 INFO:teuthology.orchestra.run.smithi204.stdout:(30/152): rbd-mirror-19.0.0-3450.g9a047fe7.el9. 3.9 MB/s | 3.1 MB 00:00 2024-05-03T20:23:07.344 INFO:teuthology.orchestra.run.smithi181.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3450.g9a047 12 MB/s | 3.5 MB 00:00 2024-05-03T20:23:07.430 INFO:teuthology.orchestra.run.smithi196.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 100 kB/s | 26 kB 00:00 2024-05-03T20:23:07.547 INFO:teuthology.orchestra.run.smithi196.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 149 kB/s | 49 kB 00:00 2024-05-03T20:23:07.739 INFO:teuthology.orchestra.run.smithi196.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 218 kB/s | 67 kB 00:00 2024-05-03T20:23:07.939 INFO:teuthology.orchestra.run.smithi196.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 386 kB/s | 151 kB 00:00 2024-05-03T20:23:07.961 INFO:teuthology.orchestra.run.smithi181.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 12 MB/s | 7.4 MB 00:00 2024-05-03T20:23:08.028 INFO:teuthology.orchestra.run.smithi181.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3450.g9a 4.0 MB/s | 274 kB 00:00 2024-05-03T20:23:08.079 INFO:teuthology.orchestra.run.smithi181.stdout:(33/152): ceph-mgr-rook-19.0.0-3450.g9a047fe7.e 986 kB/s | 49 kB 00:00 2024-05-03T20:23:08.129 INFO:teuthology.orchestra.run.smithi181.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3450.g9 328 kB/s | 16 kB 00:00 2024-05-03T20:23:08.175 INFO:teuthology.orchestra.run.smithi204.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3450.g9a047 3.9 MB/s | 3.5 MB 00:00 2024-05-03T20:23:08.179 INFO:teuthology.orchestra.run.smithi181.stdout:(35/152): ceph-volume-19.0.0-3450.g9a047fe7.el9 5.3 MB/s | 273 kB 00:00 2024-05-03T20:23:08.279 INFO:teuthology.orchestra.run.smithi181.stdout:(36/152): cephadm-19.0.0-3450.g9a047fe7.el9.noa 7.5 MB/s | 763 kB 00:00 2024-05-03T20:23:08.284 INFO:teuthology.orchestra.run.smithi204.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3450.g9a 2.5 MB/s | 274 kB 00:00 2024-05-03T20:23:08.322 INFO:teuthology.orchestra.run.smithi181.stdout:(37/152): cephfs-top-19.0.0-3450.g9a047fe7.el9. 437 kB/s | 18 kB 00:00 2024-05-03T20:23:08.331 INFO:teuthology.orchestra.run.smithi196.stdout:(112/152): libarrow-9.0.0-11.el9.x86_64.rpm 3.5 MB/s | 4.4 MB 00:01 2024-05-03T20:23:08.334 INFO:teuthology.orchestra.run.smithi204.stdout:(33/152): ceph-mgr-rook-19.0.0-3450.g9a047fe7.e 986 kB/s | 49 kB 00:00 2024-05-03T20:23:08.357 INFO:teuthology.orchestra.run.smithi196.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.3 MB/s | 839 kB 00:00 2024-05-03T20:23:08.376 INFO:teuthology.orchestra.run.smithi204.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3450.g9 396 kB/s | 16 kB 00:00 2024-05-03T20:23:08.430 INFO:teuthology.orchestra.run.smithi181.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 743 kB/s | 80 kB 00:00 2024-05-03T20:23:08.476 INFO:teuthology.orchestra.run.smithi204.stdout:(35/152): ceph-volume-19.0.0-3450.g9a047fe7.el9 2.7 MB/s | 273 kB 00:00 2024-05-03T20:23:08.498 INFO:teuthology.orchestra.run.smithi196.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 981 kB/s | 548 kB 00:00 2024-05-03T20:23:08.555 INFO:teuthology.orchestra.run.smithi181.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 803 kB 00:00 2024-05-03T20:23:08.606 INFO:teuthology.orchestra.run.smithi181.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-05-03T20:23:08.607 INFO:teuthology.orchestra.run.smithi196.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 129 kB/s | 32 kB 00:00 2024-05-03T20:23:08.627 INFO:teuthology.orchestra.run.smithi204.stdout:(36/152): cephadm-19.0.0-3450.g9a047fe7.el9.noa 5.0 MB/s | 763 kB 00:00 2024-05-03T20:23:08.649 INFO:teuthology.orchestra.run.smithi196.stdout:(116/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 137 kB/s | 43 kB 00:00 2024-05-03T20:23:08.656 INFO:teuthology.orchestra.run.smithi181.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-05-03T20:23:08.677 INFO:teuthology.orchestra.run.smithi204.stdout:(37/152): cephfs-top-19.0.0-3450.g9a047fe7.el9. 366 kB/s | 18 kB 00:00 2024-05-03T20:23:08.740 INFO:teuthology.orchestra.run.smithi181.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-05-03T20:23:08.741 INFO:teuthology.orchestra.run.smithi196.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 62 kB/s | 15 kB 00:00 2024-05-03T20:23:08.790 INFO:teuthology.orchestra.run.smithi181.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-05-03T20:23:08.811 INFO:teuthology.orchestra.run.smithi204.stdout:(38/152): ceph-mgr-diskprediction-local-19.0.0- 5.0 MB/s | 7.4 MB 00:01 2024-05-03T20:23:08.840 INFO:teuthology.orchestra.run.smithi181.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-05-03T20:23:08.882 INFO:teuthology.orchestra.run.smithi181.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-05-03T20:23:08.925 INFO:teuthology.orchestra.run.smithi181.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 5.0 MB/s | 215 kB 00:00 2024-05-03T20:23:08.928 INFO:teuthology.orchestra.run.smithi204.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 321 kB/s | 80 kB 00:00 2024-05-03T20:23:09.017 INFO:teuthology.orchestra.run.smithi196.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 420 kB/s | 172 kB 00:00 2024-05-03T20:23:09.020 INFO:teuthology.orchestra.run.smithi204.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-05-03T20:23:09.078 INFO:teuthology.orchestra.run.smithi204.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.3 MB/s | 253 kB 00:00 2024-05-03T20:23:09.100 INFO:teuthology.orchestra.run.smithi181.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-05-03T20:23:09.104 INFO:teuthology.orchestra.run.smithi204.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.7 MB/s | 803 kB 00:00 2024-05-03T20:23:09.150 INFO:teuthology.orchestra.run.smithi196.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 716 kB/s | 358 kB 00:00 2024-05-03T20:23:09.162 INFO:teuthology.orchestra.run.smithi204.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-05-03T20:23:09.175 INFO:teuthology.orchestra.run.smithi181.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 943 kB 00:00 2024-05-03T20:23:09.237 INFO:teuthology.orchestra.run.smithi204.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 7.7 MB/s | 1.2 MB 00:00 2024-05-03T20:23:09.242 INFO:teuthology.orchestra.run.smithi181.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-05-03T20:23:09.258 INFO:teuthology.orchestra.run.smithi196.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 146 kB/s | 35 kB 00:00 2024-05-03T20:23:09.263 INFO:teuthology.orchestra.run.smithi204.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-05-03T20:23:09.284 INFO:teuthology.orchestra.run.smithi181.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 514 kB/s | 21 kB 00:00 2024-05-03T20:23:09.296 INFO:teuthology.orchestra.run.smithi204.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-05-03T20:23:09.300 INFO:teuthology.orchestra.run.smithi196.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 542 kB/s | 303 kB 00:00 2024-05-03T20:23:09.322 INFO:teuthology.orchestra.run.smithi204.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-05-03T20:23:09.351 INFO:teuthology.orchestra.run.smithi181.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-05-03T20:23:09.464 INFO:teuthology.orchestra.run.smithi204.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-05-03T20:23:09.514 INFO:teuthology.orchestra.run.smithi204.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 4.8 MB/s | 943 kB 00:00 2024-05-03T20:23:09.539 INFO:teuthology.orchestra.run.smithi204.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-05-03T20:23:09.551 INFO:teuthology.orchestra.run.smithi181.stdout:(52/152): ceph-common-19.0.0-3450.g9a047fe7.el9 2.0 MB/s | 21 MB 00:10 2024-05-03T20:23:09.565 INFO:teuthology.orchestra.run.smithi204.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 428 kB/s | 21 kB 00:00 2024-05-03T20:23:09.576 INFO:teuthology.orchestra.run.smithi181.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 145 kB/s | 33 kB 00:00 2024-05-03T20:23:09.590 INFO:teuthology.orchestra.run.smithi204.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-05-03T20:23:09.601 INFO:teuthology.orchestra.run.smithi196.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 456 kB/s | 206 kB 00:00 2024-05-03T20:23:09.615 INFO:teuthology.orchestra.run.smithi204.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 653 kB/s | 33 kB 00:00 2024-05-03T20:23:09.644 INFO:teuthology.orchestra.run.smithi181.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 272 kB/s | 18 kB 00:00 2024-05-03T20:23:09.657 INFO:teuthology.orchestra.run.smithi204.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-05-03T20:23:09.701 INFO:teuthology.orchestra.run.smithi196.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 361 kB/s | 144 kB 00:00 2024-05-03T20:23:09.710 INFO:teuthology.orchestra.run.smithi181.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-05-03T20:23:09.733 INFO:teuthology.orchestra.run.smithi204.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-05-03T20:23:09.758 INFO:teuthology.orchestra.run.smithi204.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-05-03T20:23:09.760 INFO:teuthology.orchestra.run.smithi181.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-03T20:23:09.768 INFO:teuthology.orchestra.run.smithi196.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 64 kB/s | 11 kB 00:00 2024-05-03T20:23:09.783 INFO:teuthology.orchestra.run.smithi204.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-03T20:23:09.808 INFO:teuthology.orchestra.run.smithi204.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-05-03T20:23:09.819 INFO:teuthology.orchestra.run.smithi181.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 774 kB/s | 45 kB 00:00 2024-05-03T20:23:09.851 INFO:teuthology.orchestra.run.smithi204.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-05-03T20:23:09.869 INFO:teuthology.orchestra.run.smithi181.stdout:(58/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.4 MB/s | 3.0 MB 00:00 2024-05-03T20:23:09.892 INFO:teuthology.orchestra.run.smithi204.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 582 kB 00:00 2024-05-03T20:23:09.903 INFO:teuthology.orchestra.run.smithi181.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 662 kB 00:00 2024-05-03T20:23:09.918 INFO:teuthology.orchestra.run.smithi204.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 379 kB/s | 25 kB 00:00 2024-05-03T20:23:09.928 INFO:teuthology.orchestra.run.smithi181.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 582 kB 00:00 2024-05-03T20:23:09.943 INFO:teuthology.orchestra.run.smithi204.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 938 kB/s | 47 kB 00:00 2024-05-03T20:23:09.951 INFO:teuthology.orchestra.run.smithi196.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 71 kB/s | 18 kB 00:00 2024-05-03T20:23:09.953 INFO:teuthology.orchestra.run.smithi181.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 506 kB/s | 25 kB 00:00 2024-05-03T20:23:09.969 INFO:teuthology.orchestra.run.smithi204.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-03T20:23:09.979 INFO:teuthology.orchestra.run.smithi181.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-05-03T20:23:10.002 INFO:teuthology.orchestra.run.smithi204.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-05-03T20:23:10.010 INFO:teuthology.orchestra.run.smithi196.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 96 kB/s | 23 kB 00:00 2024-05-03T20:23:10.012 INFO:teuthology.orchestra.run.smithi181.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-05-03T20:23:10.028 INFO:teuthology.orchestra.run.smithi204.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 602 kB/s | 35 kB 00:00 2024-05-03T20:23:10.071 INFO:teuthology.orchestra.run.smithi181.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 606 kB/s | 35 kB 00:00 2024-05-03T20:23:10.121 INFO:teuthology.orchestra.run.smithi181.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 1.3 MB/s | 188 kB 00:00 2024-05-03T20:23:10.127 INFO:teuthology.orchestra.run.smithi196.stdout:(127/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.3 MB/s | 2.0 MB 00:00 2024-05-03T20:23:10.194 INFO:teuthology.orchestra.run.smithi196.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 80 kB/s | 19 kB 00:00 2024-05-03T20:23:10.253 INFO:teuthology.orchestra.run.smithi196.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 81 kB/s | 20 kB 00:00 2024-05-03T20:23:10.272 INFO:teuthology.orchestra.run.smithi181.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 24 MB/s | 4.7 MB 00:00 2024-05-03T20:23:10.364 INFO:teuthology.orchestra.run.smithi181.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-05-03T20:23:10.406 INFO:teuthology.orchestra.run.smithi181.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-05-03T20:23:10.439 INFO:teuthology.orchestra.run.smithi181.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-05-03T20:23:10.461 INFO:teuthology.orchestra.run.smithi204.stdout:(66/152): ceph-test-19.0.0-3450.g9a047fe7.el9.x 6.6 MB/s | 49 MB 00:07 2024-05-03T20:23:10.465 INFO:teuthology.orchestra.run.smithi181.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-05-03T20:23:10.519 INFO:teuthology.orchestra.run.smithi196.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 142 kB/s | 46 kB 00:00 2024-05-03T20:23:10.523 INFO:teuthology.orchestra.run.smithi181.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-05-03T20:23:10.545 INFO:teuthology.orchestra.run.smithi204.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 8.7 MB/s | 4.7 MB 00:00 2024-05-03T20:23:10.578 INFO:teuthology.orchestra.run.smithi196.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 242 kB/s | 79 kB 00:00 2024-05-03T20:23:10.604 INFO:teuthology.orchestra.run.smithi204.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 7.7 MB/s | 4.4 MB 00:00 2024-05-03T20:23:10.637 INFO:teuthology.orchestra.run.smithi204.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 9.7 MB/s | 911 kB 00:00 2024-05-03T20:23:10.640 INFO:teuthology.orchestra.run.smithi181.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 29 MB/s | 5.8 MB 00:00 2024-05-03T20:23:10.662 INFO:teuthology.orchestra.run.smithi204.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-05-03T20:23:10.666 INFO:teuthology.orchestra.run.smithi181.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 616 kB/s | 87 kB 00:00 2024-05-03T20:23:10.713 INFO:teuthology.orchestra.run.smithi204.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-05-03T20:23:10.716 INFO:teuthology.orchestra.run.smithi181.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-05-03T20:23:10.755 INFO:teuthology.orchestra.run.smithi204.stdout:(72/152): postgresql-13.14-1.el9.x86_64.rpm 5.4 MB/s | 1.6 MB 00:00 2024-05-03T20:23:10.762 INFO:teuthology.orchestra.run.smithi196.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 1.6 MB/s | 1.0 MB 00:00 2024-05-03T20:23:10.766 INFO:teuthology.orchestra.run.smithi181.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-05-03T20:23:10.780 INFO:teuthology.orchestra.run.smithi204.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-05-03T20:23:10.854 INFO:teuthology.orchestra.run.smithi196.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 173 kB/s | 58 kB 00:00 2024-05-03T20:23:10.855 INFO:teuthology.orchestra.run.smithi204.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 27 MB/s | 5.8 MB 00:00 2024-05-03T20:23:10.858 INFO:teuthology.orchestra.run.smithi181.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 6.0 MB 00:00 2024-05-03T20:23:10.880 INFO:teuthology.orchestra.run.smithi204.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-05-03T20:23:10.885 INFO:teuthology.orchestra.run.smithi181.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 219 kB/s | 26 kB 00:00 2024-05-03T20:23:10.906 INFO:teuthology.orchestra.run.smithi204.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-05-03T20:23:10.909 INFO:teuthology.orchestra.run.smithi181.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-05-03T20:23:10.931 INFO:teuthology.orchestra.run.smithi204.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 509 kB/s | 26 kB 00:00 2024-05-03T20:23:10.934 INFO:teuthology.orchestra.run.smithi181.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-05-03T20:23:10.951 INFO:teuthology.orchestra.run.smithi181.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 829 kB/s | 35 kB 00:00 2024-05-03T20:23:10.956 INFO:teuthology.orchestra.run.smithi204.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-05-03T20:23:10.982 INFO:teuthology.orchestra.run.smithi204.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-05-03T20:23:11.007 INFO:teuthology.orchestra.run.smithi204.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-05-03T20:23:11.029 INFO:teuthology.orchestra.run.smithi196.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 62 kB/s | 16 kB 00:00 2024-05-03T20:23:11.034 INFO:teuthology.orchestra.run.smithi181.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-05-03T20:23:11.057 INFO:teuthology.orchestra.run.smithi196.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 573 kB/s | 272 kB 00:00 2024-05-03T20:23:11.108 INFO:teuthology.orchestra.run.smithi204.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-05-03T20:23:11.110 INFO:teuthology.orchestra.run.smithi181.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 5.1 MB 00:00 2024-05-03T20:23:11.135 INFO:teuthology.orchestra.run.smithi181.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-05-03T20:23:11.160 INFO:teuthology.orchestra.run.smithi181.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-03T20:23:11.171 INFO:teuthology.orchestra.run.smithi196.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 284 kB/s | 90 kB 00:00 2024-05-03T20:23:11.183 INFO:teuthology.orchestra.run.smithi204.stdout:(82/152): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-05-03T20:23:11.186 INFO:teuthology.orchestra.run.smithi181.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 839 kB/s | 42 kB 00:00 2024-05-03T20:23:11.208 INFO:teuthology.orchestra.run.smithi204.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-05-03T20:23:11.211 INFO:teuthology.orchestra.run.smithi181.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-05-03T20:23:11.233 INFO:teuthology.orchestra.run.smithi204.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-03T20:23:11.262 INFO:teuthology.orchestra.run.smithi204.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-05-03T20:23:11.262 INFO:teuthology.orchestra.run.smithi181.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-05-03T20:23:11.284 INFO:teuthology.orchestra.run.smithi204.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-05-03T20:23:11.286 INFO:teuthology.orchestra.run.smithi181.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-05-03T20:23:11.305 INFO:teuthology.orchestra.run.smithi196.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 102 kB/s | 28 kB 00:00 2024-05-03T20:23:11.309 INFO:teuthology.orchestra.run.smithi204.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-03T20:23:11.312 INFO:teuthology.orchestra.run.smithi181.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-03T20:23:11.330 INFO:teuthology.orchestra.run.smithi196.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 111 kB/s | 31 kB 00:00 2024-05-03T20:23:11.343 INFO:teuthology.orchestra.run.smithi204.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-05-03T20:23:11.345 INFO:teuthology.orchestra.run.smithi181.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 915 kB/s | 54 kB 00:00 2024-05-03T20:23:11.369 INFO:teuthology.orchestra.run.smithi204.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 866 kB/s | 51 kB 00:00 2024-05-03T20:23:11.394 INFO:teuthology.orchestra.run.smithi204.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-03T20:23:11.446 INFO:teuthology.orchestra.run.smithi181.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-05-03T20:23:11.461 INFO:teuthology.orchestra.run.smithi204.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-05-03T20:23:11.486 INFO:teuthology.orchestra.run.smithi204.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 453 kB/s | 42 kB 00:00 2024-05-03T20:23:11.504 INFO:teuthology.orchestra.run.smithi181.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 757 kB/s | 44 kB 00:00 2024-05-03T20:23:11.511 INFO:teuthology.orchestra.run.smithi204.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 880 kB/s | 44 kB 00:00 2024-05-03T20:23:11.537 INFO:teuthology.orchestra.run.smithi204.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-03T20:23:11.554 INFO:teuthology.orchestra.run.smithi181.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-03T20:23:11.562 INFO:teuthology.orchestra.run.smithi204.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-05-03T20:23:11.587 INFO:teuthology.orchestra.run.smithi204.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-05-03T20:23:11.588 INFO:teuthology.orchestra.run.smithi196.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 138 kB/s | 36 kB 00:00 2024-05-03T20:23:11.613 INFO:teuthology.orchestra.run.smithi204.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-05-03T20:23:11.613 INFO:teuthology.orchestra.run.smithi181.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 615 kB/s | 36 kB 00:00 2024-05-03T20:23:11.630 INFO:teuthology.orchestra.run.smithi196.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 182 kB/s | 59 kB 00:00 2024-05-03T20:23:11.638 INFO:teuthology.orchestra.run.smithi204.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-03T20:23:11.656 INFO:teuthology.orchestra.run.smithi196.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 389 kB/s | 188 kB 00:00 2024-05-03T20:23:11.663 INFO:teuthology.orchestra.run.smithi204.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-03T20:23:11.664 INFO:teuthology.orchestra.run.smithi181.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 951 kB/s | 47 kB 00:00 2024-05-03T20:23:11.688 INFO:teuthology.orchestra.run.smithi204.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-03T20:23:11.714 INFO:teuthology.orchestra.run.smithi204.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-05-03T20:23:11.730 INFO:teuthology.orchestra.run.smithi181.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-05-03T20:23:11.739 INFO:teuthology.orchestra.run.smithi204.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 443 kB/s | 22 kB 00:00 2024-05-03T20:23:11.781 INFO:teuthology.orchestra.run.smithi204.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-05-03T20:23:11.798 INFO:teuthology.orchestra.run.smithi181.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 865 kB/s | 58 kB 00:00 2024-05-03T20:23:11.906 INFO:teuthology.orchestra.run.smithi196.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 161 kB/s | 51 kB 00:00 2024-05-03T20:23:11.998 INFO:teuthology.orchestra.run.smithi181.stdout:(98/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-05-03T20:23:12.023 INFO:teuthology.orchestra.run.smithi181.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 416 kB/s | 94 kB 00:00 2024-05-03T20:23:12.031 INFO:teuthology.orchestra.run.smithi204.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-05-03T20:23:12.049 INFO:teuthology.orchestra.run.smithi181.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-03T20:23:12.056 INFO:teuthology.orchestra.run.smithi196.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 224 kB/s | 90 kB 00:00 2024-05-03T20:23:12.082 INFO:teuthology.orchestra.run.smithi181.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 766 kB/s | 45 kB 00:00 2024-05-03T20:23:12.090 INFO:teuthology.orchestra.run.smithi196.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 500 kB/s | 230 kB 00:00 2024-05-03T20:23:12.174 INFO:teuthology.orchestra.run.smithi181.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 178 kB/s | 22 kB 00:00 2024-05-03T20:23:12.290 INFO:teuthology.orchestra.run.smithi204.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 75 kB/s | 19 kB 00:00 2024-05-03T20:23:12.291 INFO:teuthology.orchestra.run.smithi196.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 85 kB/s | 20 kB 00:00 2024-05-03T20:23:12.358 INFO:teuthology.orchestra.run.smithi181.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 3.1 MB/s | 863 kB 00:00 2024-05-03T20:23:12.424 INFO:teuthology.orchestra.run.smithi204.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 801 kB/s | 548 kB 00:00 2024-05-03T20:23:12.449 INFO:teuthology.orchestra.run.smithi204.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 463 kB/s | 309 kB 00:00 2024-05-03T20:23:12.457 INFO:teuthology.orchestra.run.smithi196.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 780 kB/s | 427 kB 00:00 2024-05-03T20:23:12.499 INFO:teuthology.orchestra.run.smithi196.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 469 kB/s | 192 kB 00:00 2024-05-03T20:23:12.524 INFO:teuthology.orchestra.run.smithi204.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 259 kB/s | 26 kB 00:00 2024-05-03T20:23:12.549 INFO:teuthology.orchestra.run.smithi204.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 485 kB/s | 49 kB 00:00 2024-05-03T20:23:12.641 INFO:teuthology.orchestra.run.smithi204.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 577 kB/s | 67 kB 00:00 2024-05-03T20:23:12.667 INFO:teuthology.orchestra.run.smithi204.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-05-03T20:23:12.708 INFO:teuthology.orchestra.run.smithi181.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-05-03T20:23:12.808 INFO:teuthology.orchestra.run.smithi181.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 195 kB/s | 19 kB 00:00 2024-05-03T20:23:12.834 INFO:teuthology.orchestra.run.smithi181.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 649 kB/s | 309 kB 00:00 2024-05-03T20:23:12.925 INFO:teuthology.orchestra.run.smithi181.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 284 kB/s | 26 kB 00:00 2024-05-03T20:23:12.926 INFO:teuthology.orchestra.run.smithi204.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.9 MB/s | 839 kB 00:00 2024-05-03T20:23:12.983 INFO:teuthology.orchestra.run.smithi196.stdout:(148/152): librbd1-19.0.0-3450.g9a047fe7.el9.x8 6.5 MB/s | 3.1 MB 00:00 2024-05-03T20:23:13.017 INFO:teuthology.orchestra.run.smithi181.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 530 kB/s | 49 kB 00:00 2024-05-03T20:23:13.024 INFO:teuthology.orchestra.run.smithi196.stdout:(149/152): thrift-0.15.0-2.el9.x86_64.rpm 2.2 MB/s | 1.6 MB 00:00 2024-05-03T20:23:13.025 INFO:teuthology.orchestra.run.smithi204.stdout:(113/152): libarrow-9.0.0-11.el9.x86_64.rpm 6.0 MB/s | 4.4 MB 00:00 2024-05-03T20:23:13.051 INFO:teuthology.orchestra.run.smithi204.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 347 kB/s | 43 kB 00:00 2024-05-03T20:23:13.085 INFO:teuthology.orchestra.run.smithi204.stdout:(115/152): python3-asyncssh-2.13.2-5.el9.noarch 1.3 MB/s | 548 kB 00:00 2024-05-03T20:23:13.118 INFO:teuthology.orchestra.run.smithi181.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 675 kB/s | 67 kB 00:00 2024-05-03T20:23:13.125 INFO:teuthology.orchestra.run.smithi196.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 9.5 MB/s | 1.3 MB 00:00 2024-05-03T20:23:13.152 INFO:teuthology.orchestra.run.smithi204.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 256 kB/s | 32 kB 00:00 2024-05-03T20:23:13.185 INFO:teuthology.orchestra.run.smithi204.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 112 kB/s | 15 kB 00:00 2024-05-03T20:23:13.192 INFO:teuthology.orchestra.run.smithi196.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-05-03T20:23:13.244 INFO:teuthology.orchestra.run.smithi204.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.1 MB/s | 172 kB 00:00 2024-05-03T20:23:13.260 INFO:teuthology.orchestra.run.smithi181.stdout:(110/152): libarrow-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 4.4 MB 00:00 2024-05-03T20:23:13.285 INFO:teuthology.orchestra.run.smithi181.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 904 kB/s | 151 kB 00:00 2024-05-03T20:23:13.344 INFO:teuthology.orchestra.run.smithi204.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 1.8 MB/s | 358 kB 00:00 2024-05-03T20:23:13.395 INFO:teuthology.orchestra.run.smithi204.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.4 MB/s | 303 kB 00:00 2024-05-03T20:23:13.420 INFO:teuthology.orchestra.run.smithi204.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 201 kB/s | 35 kB 00:00 2024-05-03T20:23:13.460 INFO:teuthology.orchestra.run.smithi181.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 3.1 MB/s | 548 kB 00:00 2024-05-03T20:23:13.467 INFO:teuthology.orchestra.run.smithi196.stdout:(152/152): librados2-19.0.0-3450.g9a047fe7.el9. 3.4 MB/s | 3.4 MB 00:01 2024-05-03T20:23:13.470 INFO:teuthology.orchestra.run.smithi204.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.6 MB/s | 206 kB 00:00 2024-05-03T20:23:13.471 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-05-03T20:23:13.471 INFO:teuthology.orchestra.run.smithi196.stdout:Total 12 MB/s | 222 MB 00:18 2024-05-03T20:23:13.494 INFO:teuthology.orchestra.run.smithi181.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.5 MB/s | 839 kB 00:00 2024-05-03T20:23:13.594 INFO:teuthology.orchestra.run.smithi181.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 325 kB/s | 43 kB 00:00 2024-05-03T20:23:13.637 INFO:teuthology.orchestra.run.smithi204.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 666 kB/s | 144 kB 00:00 2024-05-03T20:23:13.644 INFO:teuthology.orchestra.run.smithi181.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 214 kB/s | 32 kB 00:00 2024-05-03T20:23:13.662 INFO:teuthology.orchestra.run.smithi204.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 56 kB/s | 11 kB 00:00 2024-05-03T20:23:13.703 INFO:teuthology.orchestra.run.smithi181.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-05-03T20:23:13.713 INFO:teuthology.orchestra.run.smithi204.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 6.4 MB/s | 2.0 MB 00:00 2024-05-03T20:23:13.738 INFO:teuthology.orchestra.run.smithi204.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-05-03T20:23:13.745 INFO:teuthology.orchestra.run.smithi181.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-05-03T20:23:13.780 INFO:teuthology.orchestra.run.smithi204.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 197 kB/s | 23 kB 00:00 2024-05-03T20:23:13.814 INFO:teuthology.orchestra.run.smithi204.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-05-03T20:23:13.820 INFO:teuthology.orchestra.run.smithi181.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-05-03T20:23:13.839 INFO:teuthology.orchestra.run.smithi204.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-05-03T20:23:13.846 INFO:teuthology.orchestra.run.smithi181.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-05-03T20:23:13.913 INFO:teuthology.orchestra.run.smithi181.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 384 kB/s | 35 kB 00:00 2024-05-03T20:23:13.939 INFO:teuthology.orchestra.run.smithi204.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-05-03T20:23:13.946 INFO:teuthology.orchestra.run.smithi181.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-05-03T20:23:13.964 INFO:teuthology.orchestra.run.smithi204.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 308 kB/s | 46 kB 00:00 2024-05-03T20:23:13.990 INFO:teuthology.orchestra.run.smithi204.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 523 kB/s | 79 kB 00:00 2024-05-03T20:23:14.063 INFO:teuthology.orchestra.run.smithi181.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-05-03T20:23:14.065 INFO:teuthology.orchestra.run.smithi204.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 460 kB/s | 58 kB 00:00 2024-05-03T20:23:14.115 INFO:teuthology.orchestra.run.smithi204.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-05-03T20:23:14.141 INFO:teuthology.orchestra.run.smithi204.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 109 kB/s | 16 kB 00:00 2024-05-03T20:23:14.147 INFO:teuthology.orchestra.run.smithi181.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-05-03T20:23:14.166 INFO:teuthology.orchestra.run.smithi204.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 894 kB/s | 90 kB 00:00 2024-05-03T20:23:14.189 INFO:teuthology.orchestra.run.smithi181.stdout:(124/152): python3-grpcio-1.46.7-10.el9.x86_64. 7.4 MB/s | 2.0 MB 00:00 2024-05-03T20:23:14.239 INFO:teuthology.orchestra.run.smithi181.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-05-03T20:23:14.281 INFO:teuthology.orchestra.run.smithi181.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 251 kB/s | 23 kB 00:00 2024-05-03T20:23:14.291 INFO:teuthology.orchestra.run.smithi204.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 161 kB/s | 28 kB 00:00 2024-05-03T20:23:14.316 INFO:teuthology.orchestra.run.smithi204.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 175 kB/s | 31 kB 00:00 2024-05-03T20:23:14.331 INFO:teuthology.orchestra.run.smithi181.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-05-03T20:23:14.342 INFO:teuthology.orchestra.run.smithi204.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.0 MB/s | 188 kB 00:00 2024-05-03T20:23:14.373 INFO:teuthology.orchestra.run.smithi181.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-05-03T20:23:14.401 INFO:teuthology.orchestra.run.smithi204.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 542 kB/s | 59 kB 00:00 2024-05-03T20:23:14.425 INFO:teuthology.orchestra.run.smithi204.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 328 kB/s | 36 kB 00:00 2024-05-03T20:23:14.459 INFO:teuthology.orchestra.run.smithi204.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 435 kB/s | 51 kB 00:00 2024-05-03T20:23:14.465 INFO:teuthology.orchestra.run.smithi181.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 506 kB/s | 46 kB 00:00 2024-05-03T20:23:14.507 INFO:teuthology.orchestra.run.smithi181.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-05-03T20:23:14.534 INFO:teuthology.orchestra.run.smithi204.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-05-03T20:23:14.558 INFO:teuthology.orchestra.run.smithi181.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 856 kB/s | 79 kB 00:00 2024-05-03T20:23:14.560 INFO:teuthology.orchestra.run.smithi204.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 668 kB/s | 90 kB 00:00 2024-05-03T20:23:14.593 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-05-03T20:23:14.625 INFO:teuthology.orchestra.run.smithi181.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 493 kB/s | 58 kB 00:00 2024-05-03T20:23:14.635 INFO:teuthology.orchestra.run.smithi204.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.4 MB/s | 427 kB 00:00 2024-05-03T20:23:14.652 INFO:teuthology.orchestra.run.smithi204.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 170 kB/s | 20 kB 00:00 2024-05-03T20:23:14.658 INFO:teuthology.orchestra.run.smithi181.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-05-03T20:23:14.694 INFO:teuthology.orchestra.run.smithi204.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-05-03T20:23:14.733 INFO:teuthology.orchestra.run.smithi181.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-05-03T20:23:14.738 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-05-03T20:23:14.738 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-05-03T20:23:14.759 INFO:teuthology.orchestra.run.smithi181.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 897 kB/s | 90 kB 00:00 2024-05-03T20:23:14.795 INFO:teuthology.orchestra.run.smithi204.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-05-03T20:23:14.826 INFO:teuthology.orchestra.run.smithi181.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 307 kB/s | 28 kB 00:00 2024-05-03T20:23:14.851 INFO:teuthology.orchestra.run.smithi181.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-05-03T20:23:14.926 INFO:teuthology.orchestra.run.smithi181.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-05-03T20:23:15.002 INFO:teuthology.orchestra.run.smithi181.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 393 kB/s | 59 kB 00:00 2024-05-03T20:23:15.035 INFO:teuthology.orchestra.run.smithi181.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 329 kB/s | 36 kB 00:00 2024-05-03T20:23:15.095 INFO:teuthology.orchestra.run.smithi204.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 4.5 MB/s | 1.3 MB 00:00 2024-05-03T20:23:15.102 INFO:teuthology.orchestra.run.smithi181.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 509 kB/s | 51 kB 00:00 2024-05-03T20:23:15.136 INFO:teuthology.orchestra.run.smithi181.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-05-03T20:23:15.153 INFO:teuthology.orchestra.run.smithi204.stdout:(150/152): librbd1-19.0.0-3450.g9a047fe7.el9.x8 6.8 MB/s | 3.1 MB 00:00 2024-05-03T20:23:15.203 INFO:teuthology.orchestra.run.smithi181.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 895 kB/s | 90 kB 00:00 2024-05-03T20:23:15.236 INFO:teuthology.orchestra.run.smithi181.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-05-03T20:23:15.270 INFO:teuthology.orchestra.run.smithi204.stdout:(151/152): librados2-19.0.0-3450.g9a047fe7.el9. 5.5 MB/s | 3.4 MB 00:00 2024-05-03T20:23:15.295 INFO:teuthology.orchestra.run.smithi181.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-05-03T20:23:15.371 INFO:teuthology.orchestra.run.smithi204.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 8.7 MB/s | 2.4 MB 00:00 2024-05-03T20:23:15.373 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-05-03T20:23:15.374 INFO:teuthology.orchestra.run.smithi204.stdout:Total 12 MB/s | 222 MB 00:17 2024-05-03T20:23:15.403 INFO:teuthology.orchestra.run.smithi181.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-05-03T20:23:15.479 INFO:teuthology.orchestra.run.smithi181.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-05-03T20:23:15.754 INFO:teuthology.orchestra.run.smithi181.stdout:(148/152): librados2-19.0.0-3450.g9a047fe7.el9. 9.7 MB/s | 3.4 MB 00:00 2024-05-03T20:23:15.987 INFO:teuthology.orchestra.run.smithi181.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 5.8 MB/s | 1.3 MB 00:00 2024-05-03T20:23:16.405 INFO:teuthology.orchestra.run.smithi181.stdout:(150/152): ceph-test-19.0.0-3450.g9a047fe7.el9. 4.1 MB/s | 49 MB 00:11 2024-05-03T20:23:16.463 INFO:teuthology.orchestra.run.smithi181.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 5.0 MB/s | 2.4 MB 00:00 2024-05-03T20:23:16.476 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-05-03T20:23:16.536 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-05-03T20:23:16.536 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-05-03T20:23:16.619 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-05-03T20:23:16.620 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-05-03T20:23:16.680 INFO:teuthology.orchestra.run.smithi181.stdout:(152/152): librbd1-19.0.0-3450.g9a047fe7.el9.x8 2.6 MB/s | 3.1 MB 00:01 2024-05-03T20:23:16.684 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-05-03T20:23:16.684 INFO:teuthology.orchestra.run.smithi181.stdout:Total 12 MB/s | 222 MB 00:18 2024-05-03T20:23:17.752 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-05-03T20:23:17.893 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-05-03T20:23:17.894 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-05-03T20:23:18.373 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-05-03T20:23:18.374 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-05-03T20:23:19.645 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-05-03T20:23:19.646 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-05-03T20:23:22.468 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-05-03T20:23:22.579 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-05-03T20:23:22.617 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-05-03T20:23:22.755 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-05-03T20:23:22.947 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-05-03T20:23:22.972 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-05-03T20:23:23.017 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-05-03T20:23:23.346 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-05-03T20:23:23.473 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T20:23:23.611 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T20:23:23.641 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T20:23:23.716 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T20:23:23.753 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T20:23:23.892 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-05-03T20:23:23.950 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-05-03T20:23:23.978 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T20:23:24.081 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T20:23:24.104 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T20:23:24.166 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T20:23:24.188 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T20:23:24.212 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T20:23:24.259 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-05-03T20:23:24.342 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-05-03T20:23:24.380 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-05-03T20:23:24.405 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T20:23:24.438 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T20:23:24.515 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-05-03T20:23:24.549 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-05-03T20:23:24.649 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-05-03T20:23:24.652 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-05-03T20:23:24.677 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-05-03T20:23:24.679 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-05-03T20:23:24.714 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-05-03T20:23:24.720 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-05-03T20:23:24.770 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-05-03T20:23:24.805 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-05-03T20:23:24.848 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-05-03T20:23:24.880 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-05-03T20:23:24.908 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-05-03T20:23:24.978 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-05-03T20:23:25.033 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-05-03T20:23:25.070 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-05-03T20:23:25.102 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-05-03T20:23:25.113 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-05-03T20:23:25.161 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T20:23:25.189 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-05-03T20:23:25.224 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 31/156 2024-05-03T20:23:25.267 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 32/156 2024-05-03T20:23:25.283 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T20:23:25.296 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 33/156 2024-05-03T20:23:25.321 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T20:23:25.357 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-05-03T20:23:25.411 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-05-03T20:23:25.412 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T20:23:25.450 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T20:23:25.484 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-05-03T20:23:25.598 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-05-03T20:23:25.618 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-05-03T20:23:25.647 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-05-03T20:23:25.671 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-05-03T20:23:25.675 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T20:23:25.697 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-05-03T20:23:25.778 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T20:23:25.784 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-05-03T20:23:25.801 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T20:23:25.822 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-05-03T20:23:25.847 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T20:23:25.869 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T20:23:25.876 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-05-03T20:23:25.890 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T20:23:25.910 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-05-03T20:23:25.936 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-05-03T20:23:25.978 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-05-03T20:23:26.086 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T20:23:26.118 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T20:23:26.202 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-05-03T20:23:26.226 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-05-03T20:23:26.284 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-05-03T20:23:26.344 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-05-03T20:23:26.344 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-05-03T20:23:26.344 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-05-03T20:23:26.351 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-05-03T20:23:26.355 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-05-03T20:23:26.393 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 43/156 2024-05-03T20:23:26.412 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T20:23:26.435 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-05-03T20:23:26.455 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-05-03T20:23:26.533 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T20:23:26.562 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T20:23:26.583 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-05-03T20:23:26.644 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T20:23:26.646 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-05-03T20:23:26.675 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T20:23:26.700 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-05-03T20:23:26.703 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-05-03T20:23:26.744 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-05-03T20:23:26.773 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-05-03T20:23:26.788 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-05-03T20:23:26.804 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-05-03T20:23:26.820 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-05-03T20:23:26.820 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-05-03T20:23:26.841 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-05-03T20:23:26.848 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-05-03T20:23:26.872 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-05-03T20:23:26.888 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-05-03T20:23:26.900 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T20:23:26.916 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-05-03T20:23:26.924 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-05-03T20:23:26.967 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-05-03T20:23:26.970 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-05-03T20:23:27.001 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-05-03T20:23:27.004 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T20:23:27.010 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-05-03T20:23:27.026 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T20:23:27.036 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-05-03T20:23:27.088 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T20:23:27.110 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T20:23:27.112 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-05-03T20:23:27.135 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T20:23:27.147 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 31/156 2024-05-03T20:23:27.153 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-05-03T20:23:27.190 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 32/156 2024-05-03T20:23:27.206 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-05-03T20:23:27.222 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 33/156 2024-05-03T20:23:27.236 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-05-03T20:23:27.278 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-05-03T20:23:27.289 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-05-03T20:23:27.309 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T20:23:27.334 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-05-03T20:23:27.343 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T20:23:27.372 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-05-03T20:23:27.409 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-05-03T20:23:27.411 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-05-03T20:23:27.450 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-05-03T20:23:27.459 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-05-03T20:23:27.510 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-05-03T20:23:27.535 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-05-03T20:23:27.545 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-05-03T20:23:27.546 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-05-03T20:23:27.584 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-05-03T20:23:27.604 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-05-03T20:23:27.609 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-05-03T20:23:27.618 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-05-03T20:23:27.683 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-05-03T20:23:27.718 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-05-03T20:23:27.736 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-05-03T20:23:27.761 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-05-03T20:23:27.775 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-05-03T20:23:27.794 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-05-03T20:23:27.803 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-05-03T20:23:27.822 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-05-03T20:23:27.837 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-05-03T20:23:27.877 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-05-03T20:23:27.890 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-05-03T20:23:27.908 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-05-03T20:23:27.938 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-05-03T20:23:27.943 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-05-03T20:23:27.973 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-05-03T20:23:27.983 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-05-03T20:23:28.006 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-05-03T20:23:28.018 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-05-03T20:23:28.047 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-05-03T20:23:28.082 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-05-03T20:23:28.095 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-05-03T20:23:28.129 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 31/156 2024-05-03T20:23:28.163 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-05-03T20:23:28.172 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 32/156 2024-05-03T20:23:28.173 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-05-03T20:23:28.201 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 33/156 2024-05-03T20:23:28.220 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-05-03T20:23:28.250 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-05-03T20:23:28.251 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-05-03T20:23:28.271 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-05-03T20:23:28.286 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-05-03T20:23:28.299 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-05-03T20:23:28.315 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-05-03T20:23:28.330 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-05-03T20:23:28.367 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 43/156 2024-05-03T20:23:28.389 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-05-03T20:23:28.397 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-05-03T20:23:28.409 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-05-03T20:23:28.444 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-05-03T20:23:28.529 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-05-03T20:23:28.565 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-05-03T20:23:28.583 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-05-03T20:23:28.620 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-05-03T20:23:28.674 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-05-03T20:23:28.747 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-05-03T20:23:28.787 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-05-03T20:23:28.824 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-05-03T20:23:28.870 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-05-03T20:23:28.906 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-05-03T20:23:28.950 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-05-03T20:23:28.991 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-05-03T20:23:29.104 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-05-03T20:23:29.127 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-05-03T20:23:29.162 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-05-03T20:23:29.172 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-05-03T20:23:29.187 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-05-03T20:23:29.203 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-05-03T20:23:29.208 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-05-03T20:23:29.217 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-05-03T20:23:29.242 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-05-03T20:23:29.244 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-05-03T20:23:29.260 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-05-03T20:23:29.298 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 43/156 2024-05-03T20:23:29.338 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-05-03T20:23:29.339 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-05-03T20:23:29.340 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-05-03T20:23:29.378 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-05-03T20:23:29.391 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-05-03T20:23:29.418 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-05-03T20:23:29.443 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-05-03T20:23:29.470 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-05-03T20:23:29.478 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-05-03T20:23:29.497 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-05-03T20:23:29.519 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-05-03T20:23:29.542 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-05-03T20:23:29.551 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-05-03T20:23:29.605 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-05-03T20:23:29.669 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-05-03T20:23:29.669 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-05-03T20:23:29.688 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-05-03T20:23:29.701 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-05-03T20:23:29.701 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-05-03T20:23:29.708 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-05-03T20:23:29.729 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-05-03T20:23:29.738 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-05-03T20:23:29.763 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-05-03T20:23:29.784 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-05-03T20:23:29.785 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-05-03T20:23:29.802 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-05-03T20:23:29.820 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-05-03T20:23:29.833 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-05-03T20:23:29.863 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-05-03T20:23:29.864 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-05-03T20:23:29.898 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-05-03T20:23:29.905 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-05-03T20:23:29.931 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-05-03T20:23:29.973 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-05-03T20:23:30.041 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-05-03T20:23:30.082 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-05-03T20:23:30.094 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-05-03T20:23:30.124 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-05-03T20:23:30.165 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-05-03T20:23:30.169 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-05-03T20:23:30.221 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-05-03T20:23:30.252 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-05-03T20:23:30.259 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-05-03T20:23:30.291 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-05-03T20:23:30.299 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-05-03T20:23:30.331 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-05-03T20:23:30.357 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-05-03T20:23:30.381 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-05-03T20:23:30.397 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-05-03T20:23:30.430 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-05-03T20:23:30.445 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-05-03T20:23:30.455 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-05-03T20:23:30.547 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-05-03T20:23:30.582 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-05-03T20:23:30.622 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-05-03T20:23:30.650 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-05-03T20:23:30.683 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-05-03T20:23:30.723 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-05-03T20:23:30.754 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-05-03T20:23:30.784 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-05-03T20:23:30.819 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-05-03T20:23:30.927 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-05-03T20:23:31.121 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-05-03T20:23:31.162 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-05-03T20:23:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-05-03T20:23:31.210 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-05-03T20:23:31.243 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-05-03T20:23:31.253 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-05-03T20:23:31.301 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-05-03T20:23:31.332 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-05-03T20:23:31.341 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-05-03T20:23:31.378 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-05-03T20:23:31.401 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-05-03T20:23:31.476 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-05-03T20:23:31.489 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-05-03T20:23:31.524 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-05-03T20:23:31.537 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-05-03T20:23:31.720 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-05-03T20:23:31.752 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-05-03T20:23:31.761 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-05-03T20:23:31.830 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-05-03T20:23:31.845 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-05-03T20:23:31.873 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-05-03T20:23:31.906 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-05-03T20:23:32.218 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-05-03T20:23:32.238 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-05-03T20:23:32.260 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T20:23:32.280 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-05-03T20:23:32.314 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-05-03T20:23:32.351 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T20:23:32.382 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 100/156 2024-05-03T20:23:32.410 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-05-03T20:23:32.417 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-05-03T20:23:32.462 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-05-03T20:23:32.471 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-05-03T20:23:32.550 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-05-03T20:23:32.590 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-05-03T20:23:32.591 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-05-03T20:23:32.767 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-05-03T20:23:32.796 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-05-03T20:23:32.874 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-05-03T20:23:32.895 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-05-03T20:23:32.921 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T20:23:32.993 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T20:23:33.006 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 104/156 2024-05-03T20:23:33.513 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-05-03T20:23:33.743 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-05-03T20:23:33.825 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-05-03T20:23:33.923 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-05-03T20:23:33.951 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-05-03T20:23:33.984 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-05-03T20:23:34.295 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-05-03T20:23:34.358 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T20:23:34.454 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T20:23:34.485 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 100/156 2024-05-03T20:23:34.521 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-05-03T20:23:34.959 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-05-03T20:23:35.000 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-05-03T20:23:35.033 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T20:23:35.049 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-05-03T20:23:35.078 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-05-03T20:23:35.087 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T20:23:35.110 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 104/156 2024-05-03T20:23:35.119 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-05-03T20:23:35.272 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T20:23:35.311 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T20:23:35.422 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-05-03T20:23:35.465 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T20:23:35.556 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T20:23:35.587 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 100/156 2024-05-03T20:23:35.623 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-05-03T20:23:35.852 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T20:23:35.897 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T20:23:35.987 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T20:23:35.987 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-03T20:23:35.987 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-03T20:23:35.988 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-03T20:23:36.016 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T20:23:36.103 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-05-03T20:23:36.135 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T20:23:36.197 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T20:23:36.220 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 104/156 2024-05-03T20:23:37.406 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T20:23:37.440 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T20:23:37.980 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T20:23:38.019 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T20:23:38.114 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T20:23:38.114 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-03T20:23:38.114 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-03T20:23:38.114 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-03T20:23:38.137 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T20:23:38.573 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T20:23:38.618 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T20:23:39.161 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T20:23:39.196 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T20:23:39.284 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T20:23:39.284 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-03T20:23:39.285 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-03T20:23:39.285 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:23:39.322 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T20:23:47.314 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T20:23:47.314 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /sys 2024-05-03T20:23:47.314 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /proc 2024-05-03T20:23:47.315 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /mnt 2024-05-03T20:23:47.315 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /var/tmp 2024-05-03T20:23:47.315 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /home 2024-05-03T20:23:47.315 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /root 2024-05-03T20:23:47.315 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /tmp 2024-05-03T20:23:47.315 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-03T20:23:47.514 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T20:23:47.530 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T20:23:47.530 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T20:23:47.530 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-03T20:23:47.530 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-03T20:23:49.025 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T20:23:49.050 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T20:23:49.051 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T20:23:49.051 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-03T20:23:49.051 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T20:23:49.051 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T20:23:49.051 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-03T20:23:49.276 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T20:23:49.298 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T20:23:49.298 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T20:23:49.298 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-03T20:23:49.298 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T20:23:49.298 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T20:23:49.299 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-03T20:23:49.743 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T20:23:49.765 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T20:23:49.765 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T20:23:49.766 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-03T20:23:49.766 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T20:23:49.766 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T20:23:49.766 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-03T20:23:49.785 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T20:23:49.785 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /sys 2024-05-03T20:23:49.785 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /proc 2024-05-03T20:23:49.785 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /mnt 2024-05-03T20:23:49.785 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /var/tmp 2024-05-03T20:23:49.785 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /home 2024-05-03T20:23:49.785 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /root 2024-05-03T20:23:49.785 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /tmp 2024-05-03T20:23:49.785 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-03T20:23:49.872 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-05-03T20:23:49.921 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-05-03T20:23:49.963 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T20:23:49.978 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-05-03T20:23:49.982 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T20:23:49.982 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T20:23:49.982 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-03T20:23:49.982 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-03T20:23:50.163 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-05-03T20:23:50.218 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-05-03T20:23:50.304 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-05-03T20:23:50.420 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T20:23:50.558 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T20:23:50.602 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-05-03T20:23:50.634 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-05-03T20:23:50.670 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-05-03T20:23:50.957 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T20:23:50.957 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /sys 2024-05-03T20:23:50.957 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /proc 2024-05-03T20:23:50.957 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /mnt 2024-05-03T20:23:50.957 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /var/tmp 2024-05-03T20:23:50.957 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /home 2024-05-03T20:23:50.957 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /root 2024-05-03T20:23:50.958 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /tmp 2024-05-03T20:23:50.958 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:23:51.122 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T20:23:51.151 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T20:23:51.162 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T20:23:51.178 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T20:23:51.178 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T20:23:51.178 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-03T20:23:51.178 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:23:51.184 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 124/156 2024-05-03T20:23:51.245 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 125/156 2024-05-03T20:23:51.275 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T20:23:51.458 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T20:23:51.484 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T20:23:51.484 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T20:23:51.484 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-03T20:23:51.484 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T20:23:51.484 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T20:23:51.484 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-03T20:23:51.708 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T20:23:51.731 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T20:23:51.731 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T20:23:51.731 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-03T20:23:51.731 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T20:23:51.731 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T20:23:51.731 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-03T20:23:52.176 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T20:23:52.199 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T20:23:52.199 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T20:23:52.199 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-03T20:23:52.199 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T20:23:52.199 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T20:23:52.199 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-03T20:23:52.298 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-05-03T20:23:52.313 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T20:23:52.351 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T20:23:52.355 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-05-03T20:23:52.418 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-05-03T20:23:52.605 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-05-03T20:23:52.649 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T20:23:52.652 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-05-03T20:23:52.674 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T20:23:52.675 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T20:23:52.675 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-03T20:23:52.675 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T20:23:52.675 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T20:23:52.675 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:23:52.730 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-05-03T20:23:52.827 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T20:23:52.900 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T20:23:52.922 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T20:23:52.923 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T20:23:52.923 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-03T20:23:52.923 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T20:23:52.923 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T20:23:52.923 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:23:52.967 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T20:23:53.011 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-05-03T20:23:53.043 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-05-03T20:23:53.070 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-05-03T20:23:53.325 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T20:23:53.352 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T20:23:53.367 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T20:23:53.390 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T20:23:53.390 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T20:23:53.390 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-03T20:23:53.390 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T20:23:53.390 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T20:23:53.390 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:23:53.464 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T20:23:53.505 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-05-03T20:23:53.506 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T20:23:53.536 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T20:23:53.554 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-05-03T20:23:53.568 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 124/156 2024-05-03T20:23:53.610 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-05-03T20:23:53.638 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 125/156 2024-05-03T20:23:53.638 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 129/156 2024-05-03T20:23:53.662 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T20:23:53.667 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T20:23:53.685 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T20:23:53.685 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T20:23:53.685 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-03T20:23:53.685 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T20:23:53.685 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T20:23:53.685 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-03T20:23:53.729 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T20:23:53.741 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T20:23:53.795 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-05-03T20:23:53.838 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 132/156 2024-05-03T20:23:53.851 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-05-03T20:23:53.886 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 133/156 2024-05-03T20:23:53.928 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-05-03T20:23:54.018 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T20:23:54.157 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T20:23:54.193 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-05-03T20:23:54.225 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-05-03T20:23:54.253 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-05-03T20:23:54.672 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T20:23:54.701 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T20:23:54.712 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T20:23:54.733 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 124/156 2024-05-03T20:23:54.744 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T20:23:54.803 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 125/156 2024-05-03T20:23:54.813 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-05-03T20:23:54.840 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T20:23:54.841 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T20:23:54.863 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T20:23:54.863 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T20:23:54.863 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-03T20:23:54.863 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T20:23:54.863 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T20:23:54.863 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-03T20:23:54.908 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T20:23:54.930 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T20:23:54.930 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T20:23:54.930 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-03T20:23:54.930 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-03T20:23:54.975 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T20:23:54.997 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T20:23:54.997 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T20:23:54.997 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-03T20:23:54.998 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T20:23:54.998 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T20:23:54.998 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-03T20:23:55.300 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T20:23:55.323 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T20:23:55.323 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T20:23:55.323 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-03T20:23:55.323 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T20:23:55.323 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T20:23:55.323 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-03T20:23:55.708 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T20:23:55.736 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T20:23:55.849 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T20:23:55.874 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T20:23:55.918 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T20:23:56.006 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 129/156 2024-05-03T20:23:56.038 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T20:23:56.063 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T20:23:56.063 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T20:23:56.063 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-03T20:23:56.063 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T20:23:56.063 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T20:23:56.063 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-03T20:23:56.122 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T20:23:56.134 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T20:23:56.231 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 132/156 2024-05-03T20:23:56.279 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 133/156 2024-05-03T20:23:56.880 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T20:23:56.910 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T20:23:57.019 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T20:23:57.182 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 129/156 2024-05-03T20:23:57.204 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-05-03T20:23:57.212 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T20:23:57.233 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T20:23:57.236 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T20:23:57.236 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T20:23:57.236 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-03T20:23:57.236 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T20:23:57.236 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T20:23:57.236 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:23:57.257 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T20:23:57.257 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T20:23:57.257 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-03T20:23:57.257 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T20:23:57.257 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T20:23:57.257 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-03T20:23:57.279 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T20:23:57.291 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T20:23:57.301 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T20:23:57.322 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T20:23:57.323 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T20:23:57.323 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-03T20:23:57.323 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-03T20:23:57.368 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T20:23:57.390 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 132/156 2024-05-03T20:23:57.390 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T20:23:57.391 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T20:23:57.391 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-03T20:23:57.391 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T20:23:57.391 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T20:23:57.391 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-03T20:23:57.436 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 133/156 2024-05-03T20:23:57.702 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T20:23:57.725 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T20:23:57.725 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T20:23:57.725 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-03T20:23:57.725 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T20:23:57.725 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T20:23:57.725 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-03T20:23:58.370 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-05-03T20:23:58.416 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T20:23:58.438 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T20:23:58.438 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T20:23:58.439 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-03T20:23:58.439 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T20:23:58.439 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T20:23:58.439 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:23:58.483 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T20:23:58.505 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T20:23:58.505 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T20:23:58.505 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-03T20:23:58.505 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:23:58.550 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T20:23:58.573 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T20:23:58.573 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T20:23:58.573 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-03T20:23:58.573 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T20:23:58.573 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T20:23:58.573 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:23:58.876 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T20:23:58.899 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T20:23:58.899 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T20:23:58.899 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-03T20:23:58.899 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T20:23:58.899 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T20:23:58.899 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:23:59.945 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 139/156 2024-05-03T20:23:59.986 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 140/156 2024-05-03T20:24:00.035 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 141/156 2024-05-03T20:24:00.068 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-05-03T20:24:00.095 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-05-03T20:24:00.275 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 144/156 2024-05-03T20:24:00.324 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-05-03T20:24:00.354 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 146/156 2024-05-03T20:24:00.942 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T20:24:00.977 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T20:24:00.996 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T20:24:01.821 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 148/156 2024-05-03T20:24:01.915 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-05-03T20:24:01.956 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-05-03T20:24:02.039 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-05-03T20:24:02.069 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-05-03T20:24:02.069 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T20:24:02.153 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T20:24:02.177 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T20:24:02.178 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T20:24:02.264 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T20:24:02.334 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T20:24:02.339 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 139/156 2024-05-03T20:24:02.381 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 140/156 2024-05-03T20:24:02.421 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 141/156 2024-05-03T20:24:02.454 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-05-03T20:24:02.481 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-05-03T20:24:02.666 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 144/156 2024-05-03T20:24:02.711 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-05-03T20:24:02.740 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 146/156 2024-05-03T20:24:03.330 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T20:24:03.372 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T20:24:03.392 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T20:24:03.522 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 139/156 2024-05-03T20:24:03.563 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 140/156 2024-05-03T20:24:03.612 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 141/156 2024-05-03T20:24:03.644 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-05-03T20:24:03.680 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-05-03T20:24:03.862 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 144/156 2024-05-03T20:24:03.901 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-05-03T20:24:03.931 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 146/156 2024-05-03T20:24:04.245 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 148/156 2024-05-03T20:24:04.335 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-05-03T20:24:04.376 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-05-03T20:24:04.458 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-05-03T20:24:04.490 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-05-03T20:24:04.490 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T20:24:04.519 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T20:24:04.562 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T20:24:04.574 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T20:24:04.581 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T20:24:04.615 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T20:24:04.615 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T20:24:04.713 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T20:24:04.780 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T20:24:05.418 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 148/156 2024-05-03T20:24:05.501 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-05-03T20:24:05.541 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-05-03T20:24:05.630 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-05-03T20:24:05.671 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-05-03T20:24:05.671 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T20:24:05.747 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T20:24:05.772 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T20:24:05.772 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T20:24:05.872 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T20:24:05.936 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T20:24:08.009 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T20:24:08.009 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 1/156 2024-05-03T20:24:08.009 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2/156 2024-05-03T20:24:08.009 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 3/156 2024-05-03T20:24:08.010 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 4/156 2024-05-03T20:24:08.010 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 5/156 2024-05-03T20:24:08.010 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 6/156 2024-05-03T20:24:08.010 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 7/156 2024-05-03T20:24:08.010 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T20:24:08.010 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T20:24:08.010 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T20:24:08.010 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 11/156 2024-05-03T20:24:08.010 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 12/156 2024-05-03T20:24:08.010 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T20:24:08.010 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 14/156 2024-05-03T20:24:08.010 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 15/156 2024-05-03T20:24:08.010 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 16/156 2024-05-03T20:24:08.011 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 17/156 2024-05-03T20:24:08.011 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 18/156 2024-05-03T20:24:08.011 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 19/156 2024-05-03T20:24:08.011 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 20/156 2024-05-03T20:24:08.011 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 21/156 2024-05-03T20:24:08.011 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 22/156 2024-05-03T20:24:08.011 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 23/156 2024-05-03T20:24:08.011 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 24/156 2024-05-03T20:24:08.011 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 25/156 2024-05-03T20:24:08.011 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 26/156 2024-05-03T20:24:08.011 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 27/156 2024-05-03T20:24:08.011 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 28/156 2024-05-03T20:24:08.011 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 29/156 2024-05-03T20:24:08.023 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 30/156 2024-05-03T20:24:08.023 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 31/156 2024-05-03T20:24:08.023 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 32/156 2024-05-03T20:24:08.024 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 33/156 2024-05-03T20:24:08.024 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 34/156 2024-05-03T20:24:08.024 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 35/156 2024-05-03T20:24:08.024 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 36/156 2024-05-03T20:24:08.024 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 37/156 2024-05-03T20:24:08.024 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 38/156 2024-05-03T20:24:08.024 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 39/156 2024-05-03T20:24:08.024 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-05-03T20:24:08.024 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-05-03T20:24:08.024 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-05-03T20:24:08.024 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-05-03T20:24:08.024 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-05-03T20:24:08.024 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-05-03T20:24:08.024 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-05-03T20:24:08.025 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-05-03T20:24:08.025 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-05-03T20:24:08.025 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-05-03T20:24:08.025 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-05-03T20:24:08.025 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-05-03T20:24:08.025 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-05-03T20:24:08.025 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-05-03T20:24:08.025 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-05-03T20:24:08.025 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-05-03T20:24:08.025 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-05-03T20:24:08.025 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-05-03T20:24:08.025 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-05-03T20:24:08.025 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-05-03T20:24:08.025 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-05-03T20:24:08.025 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-05-03T20:24:08.025 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-05-03T20:24:08.026 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-05-03T20:24:08.026 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-05-03T20:24:08.026 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-05-03T20:24:08.026 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-05-03T20:24:08.026 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-05-03T20:24:08.026 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-05-03T20:24:08.026 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-05-03T20:24:08.026 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-05-03T20:24:08.026 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-05-03T20:24:08.026 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-05-03T20:24:08.026 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-05-03T20:24:08.026 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-05-03T20:24:08.027 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-05-03T20:24:08.027 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-05-03T20:24:08.027 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-05-03T20:24:08.027 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-05-03T20:24:08.027 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-05-03T20:24:08.027 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-05-03T20:24:08.027 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-05-03T20:24:08.027 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-05-03T20:24:08.027 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-05-03T20:24:08.027 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-05-03T20:24:08.027 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-05-03T20:24:08.027 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-05-03T20:24:08.027 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-05-03T20:24:08.027 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-05-03T20:24:08.027 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-05-03T20:24:08.028 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-05-03T20:24:08.028 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-05-03T20:24:08.028 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-05-03T20:24:08.028 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-05-03T20:24:08.028 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-05-03T20:24:08.028 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-05-03T20:24:08.028 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-05-03T20:24:08.028 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-05-03T20:24:08.028 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-05-03T20:24:08.028 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-05-03T20:24:08.028 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-05-03T20:24:08.028 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-05-03T20:24:08.028 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-05-03T20:24:08.029 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-05-03T20:24:08.029 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-05-03T20:24:08.029 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-05-03T20:24:08.029 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-05-03T20:24:08.029 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-05-03T20:24:08.029 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-05-03T20:24:08.029 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-05-03T20:24:08.029 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-05-03T20:24:08.029 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-05-03T20:24:08.029 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-05-03T20:24:08.029 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-05-03T20:24:08.029 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-05-03T20:24:08.029 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-05-03T20:24:08.029 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-05-03T20:24:08.029 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-05-03T20:24:08.030 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T20:24:08.030 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T20:24:08.030 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-05-03T20:24:08.030 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-05-03T20:24:08.030 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-05-03T20:24:08.030 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-05-03T20:24:08.030 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-05-03T20:24:08.030 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-05-03T20:24:08.030 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-05-03T20:24:08.030 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-05-03T20:24:08.030 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-05-03T20:24:08.030 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-05-03T20:24:08.030 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-05-03T20:24:08.031 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-05-03T20:24:08.031 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-05-03T20:24:08.031 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-05-03T20:24:08.031 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-05-03T20:24:08.031 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-05-03T20:24:08.031 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-05-03T20:24:08.031 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-05-03T20:24:08.031 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-05-03T20:24:08.031 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-05-03T20:24:08.031 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-05-03T20:24:08.031 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-05-03T20:24:08.032 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-05-03T20:24:08.032 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-05-03T20:24:08.032 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-05-03T20:24:08.032 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-05-03T20:24:08.032 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-05-03T20:24:08.032 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-05-03T20:24:08.032 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-05-03T20:24:08.032 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 149/156 2024-05-03T20:24:08.032 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-05-03T20:24:08.032 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 151/156 2024-05-03T20:24:08.032 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-05-03T20:24:08.032 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-05-03T20:24:08.032 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T20:24:08.032 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-05-03T20:24:10.157 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T20:24:10.157 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-03T20:24:10.157 INFO:teuthology.orchestra.run.smithi196.stdout:Upgraded: 2024-05-03T20:24:10.157 INFO:teuthology.orchestra.run.smithi196.stdout: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.157 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.158 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-03T20:24:10.158 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-03T20:24:10.158 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-05-03T20:24:10.158 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-03T20:24:10.158 INFO:teuthology.orchestra.run.smithi196.stdout: automake-1.16.2-8.el9.noarch 2024-05-03T20:24:10.158 INFO:teuthology.orchestra.run.smithi196.stdout: bison-3.7.4-5.el9.x86_64 2024-05-03T20:24:10.158 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-03T20:24:10.158 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.158 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.158 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.158 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.158 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:10.158 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.159 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.159 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.159 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:10.159 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:10.159 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:10.159 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:10.159 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:10.159 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.159 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.159 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:10.159 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.159 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.159 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.159 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:10.160 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:10.160 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.160 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.160 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:10.160 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-03T20:24:10.160 INFO:teuthology.orchestra.run.smithi196.stdout: flex-2.6.4-9.el9.x86_64 2024-05-03T20:24:10.160 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-03T20:24:10.160 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-03T20:24:10.160 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-03T20:24:10.160 INFO:teuthology.orchestra.run.smithi196.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-03T20:24:10.160 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-03T20:24:10.160 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-03T20:24:10.160 INFO:teuthology.orchestra.run.smithi196.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-03T20:24:10.160 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-03T20:24:10.160 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-03T20:24:10.161 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.161 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.161 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.161 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-03T20:24:10.161 INFO:teuthology.orchestra.run.smithi196.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-03T20:24:10.161 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-03T20:24:10.161 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-03T20:24:10.161 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-03T20:24:10.161 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.161 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.161 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-03T20:24:10.161 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.161 INFO:teuthology.orchestra.run.smithi196.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-03T20:24:10.161 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-03T20:24:10.161 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-03T20:24:10.161 INFO:teuthology.orchestra.run.smithi196.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-03T20:24:10.161 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-03T20:24:10.162 INFO:teuthology.orchestra.run.smithi196.stdout: lua-5.4.4-4.el9.x86_64 2024-05-03T20:24:10.162 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-03T20:24:10.162 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-03T20:24:10.162 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-03T20:24:10.162 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-03T20:24:10.162 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-03T20:24:10.162 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-03T20:24:10.162 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-13.14-1.el9.x86_64 2024-05-03T20:24:10.162 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-03T20:24:10.162 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-03T20:24:10.162 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-03T20:24:10.162 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-03T20:24:10.162 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-03T20:24:10.162 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-03T20:24:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-03T20:24:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-03T20:24:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-03T20:24:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-03T20:24:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-03T20:24:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-03T20:24:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-03T20:24:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-03T20:24:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-03T20:24:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-03T20:24:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-03T20:24:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-03T20:24:10.163 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-03T20:24:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-03T20:24:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-03T20:24:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-03T20:24:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-03T20:24:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-03T20:24:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-03T20:24:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-03T20:24:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-03T20:24:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-03T20:24:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-03T20:24:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-03T20:24:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-03T20:24:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-03T20:24:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-03T20:24:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-03T20:24:10.164 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-03T20:24:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-03T20:24:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-03T20:24:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-03T20:24:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-03T20:24:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-03T20:24:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply-3.11-14.el9.noarch 2024-05-03T20:24:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-03T20:24:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-03T20:24:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-03T20:24:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-03T20:24:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-03T20:24:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-03T20:24:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-03T20:24:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-03T20:24:10.165 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-03T20:24:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-03T20:24:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-03T20:24:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-03T20:24:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-03T20:24:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-03T20:24:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-03T20:24:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-03T20:24:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-03T20:24:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-03T20:24:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-03T20:24:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-03T20:24:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-03T20:24:10.166 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-03T20:24:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-03T20:24:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-03T20:24:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-03T20:24:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-03T20:24:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-03T20:24:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: re2-1:20211101-3.el9.x86_64 2024-05-03T20:24:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-03T20:24:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-03T20:24:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-03T20:24:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-03T20:24:10.167 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-03T20:24:10.168 INFO:teuthology.orchestra.run.smithi196.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-03T20:24:10.168 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-05-03T20:24:10.168 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-05-03T20:24:10.544 DEBUG:teuthology.parallel:result is None 2024-05-03T20:24:10.648 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T20:24:10.648 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 1/156 2024-05-03T20:24:10.648 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2/156 2024-05-03T20:24:10.648 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 3/156 2024-05-03T20:24:10.649 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 4/156 2024-05-03T20:24:10.649 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 5/156 2024-05-03T20:24:10.649 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 6/156 2024-05-03T20:24:10.649 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 7/156 2024-05-03T20:24:10.649 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T20:24:10.649 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T20:24:10.649 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T20:24:10.649 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 11/156 2024-05-03T20:24:10.649 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 12/156 2024-05-03T20:24:10.649 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T20:24:10.649 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 14/156 2024-05-03T20:24:10.649 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 15/156 2024-05-03T20:24:10.649 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 16/156 2024-05-03T20:24:10.649 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 17/156 2024-05-03T20:24:10.649 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 18/156 2024-05-03T20:24:10.650 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 19/156 2024-05-03T20:24:10.650 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 20/156 2024-05-03T20:24:10.650 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 21/156 2024-05-03T20:24:10.650 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 22/156 2024-05-03T20:24:10.650 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 23/156 2024-05-03T20:24:10.650 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 24/156 2024-05-03T20:24:10.650 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 25/156 2024-05-03T20:24:10.650 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 26/156 2024-05-03T20:24:10.650 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 27/156 2024-05-03T20:24:10.650 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 28/156 2024-05-03T20:24:10.663 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 29/156 2024-05-03T20:24:10.663 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 30/156 2024-05-03T20:24:10.663 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 31/156 2024-05-03T20:24:10.664 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 32/156 2024-05-03T20:24:10.664 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 33/156 2024-05-03T20:24:10.664 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 34/156 2024-05-03T20:24:10.664 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 35/156 2024-05-03T20:24:10.664 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 36/156 2024-05-03T20:24:10.664 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 37/156 2024-05-03T20:24:10.664 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 38/156 2024-05-03T20:24:10.664 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 39/156 2024-05-03T20:24:10.664 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-05-03T20:24:10.664 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-05-03T20:24:10.664 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-05-03T20:24:10.664 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-05-03T20:24:10.664 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-05-03T20:24:10.664 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-05-03T20:24:10.665 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-05-03T20:24:10.665 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-05-03T20:24:10.665 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-05-03T20:24:10.665 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-05-03T20:24:10.665 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-05-03T20:24:10.665 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-05-03T20:24:10.665 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-05-03T20:24:10.665 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-05-03T20:24:10.665 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-05-03T20:24:10.665 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-05-03T20:24:10.665 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-05-03T20:24:10.665 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-05-03T20:24:10.665 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-05-03T20:24:10.666 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-05-03T20:24:10.666 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-05-03T20:24:10.666 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-05-03T20:24:10.666 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-05-03T20:24:10.666 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-05-03T20:24:10.666 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-05-03T20:24:10.666 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-05-03T20:24:10.666 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-05-03T20:24:10.666 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-05-03T20:24:10.666 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-05-03T20:24:10.666 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-05-03T20:24:10.666 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-05-03T20:24:10.666 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-05-03T20:24:10.667 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-05-03T20:24:10.667 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-05-03T20:24:10.667 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-05-03T20:24:10.667 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-05-03T20:24:10.667 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-05-03T20:24:10.667 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-05-03T20:24:10.667 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-05-03T20:24:10.667 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-05-03T20:24:10.667 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-05-03T20:24:10.667 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-05-03T20:24:10.667 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-05-03T20:24:10.667 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-05-03T20:24:10.667 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-05-03T20:24:10.667 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-05-03T20:24:10.668 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-05-03T20:24:10.668 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-05-03T20:24:10.668 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-05-03T20:24:10.668 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-05-03T20:24:10.668 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-05-03T20:24:10.668 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-05-03T20:24:10.668 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-05-03T20:24:10.668 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-05-03T20:24:10.668 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-05-03T20:24:10.668 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-05-03T20:24:10.668 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-05-03T20:24:10.668 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-05-03T20:24:10.668 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-05-03T20:24:10.668 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-05-03T20:24:10.669 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-05-03T20:24:10.669 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-05-03T20:24:10.669 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-05-03T20:24:10.669 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-05-03T20:24:10.669 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-05-03T20:24:10.669 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-05-03T20:24:10.669 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-05-03T20:24:10.669 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-05-03T20:24:10.669 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-05-03T20:24:10.669 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-05-03T20:24:10.669 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-05-03T20:24:10.669 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-05-03T20:24:10.669 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-05-03T20:24:10.669 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-05-03T20:24:10.669 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-05-03T20:24:10.669 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-05-03T20:24:10.670 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-05-03T20:24:10.670 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-05-03T20:24:10.670 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T20:24:10.670 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T20:24:10.670 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-05-03T20:24:10.670 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-05-03T20:24:10.670 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-05-03T20:24:10.670 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-05-03T20:24:10.670 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-05-03T20:24:10.670 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-05-03T20:24:10.670 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-05-03T20:24:10.670 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-05-03T20:24:10.670 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-05-03T20:24:10.670 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-05-03T20:24:10.670 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-05-03T20:24:10.671 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-05-03T20:24:10.671 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-05-03T20:24:10.671 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-05-03T20:24:10.671 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-05-03T20:24:10.671 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-05-03T20:24:10.671 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-05-03T20:24:10.671 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-05-03T20:24:10.671 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-05-03T20:24:10.671 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-05-03T20:24:10.671 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-05-03T20:24:10.671 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-05-03T20:24:10.671 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-05-03T20:24:10.671 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-05-03T20:24:10.671 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-05-03T20:24:10.671 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-05-03T20:24:10.671 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-05-03T20:24:10.672 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-05-03T20:24:10.672 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-05-03T20:24:10.672 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 149/156 2024-05-03T20:24:10.672 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-05-03T20:24:10.672 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 151/156 2024-05-03T20:24:10.672 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-05-03T20:24:10.672 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-05-03T20:24:10.672 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T20:24:10.672 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-05-03T20:24:11.914 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T20:24:11.914 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 1/156 2024-05-03T20:24:11.914 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2/156 2024-05-03T20:24:11.914 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 3/156 2024-05-03T20:24:11.915 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 4/156 2024-05-03T20:24:11.915 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 5/156 2024-05-03T20:24:11.915 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 6/156 2024-05-03T20:24:11.915 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 7/156 2024-05-03T20:24:11.915 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T20:24:11.915 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T20:24:11.915 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T20:24:11.915 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 11/156 2024-05-03T20:24:11.915 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 12/156 2024-05-03T20:24:11.915 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T20:24:11.915 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 14/156 2024-05-03T20:24:11.915 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 15/156 2024-05-03T20:24:11.915 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 16/156 2024-05-03T20:24:11.915 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 17/156 2024-05-03T20:24:11.916 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 18/156 2024-05-03T20:24:11.916 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 19/156 2024-05-03T20:24:11.916 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 20/156 2024-05-03T20:24:11.916 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 21/156 2024-05-03T20:24:11.916 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 22/156 2024-05-03T20:24:11.916 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 23/156 2024-05-03T20:24:11.916 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 24/156 2024-05-03T20:24:11.916 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 25/156 2024-05-03T20:24:11.916 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 26/156 2024-05-03T20:24:11.916 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 27/156 2024-05-03T20:24:11.916 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 28/156 2024-05-03T20:24:11.916 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 29/156 2024-05-03T20:24:11.916 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 30/156 2024-05-03T20:24:11.916 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 31/156 2024-05-03T20:24:11.917 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 32/156 2024-05-03T20:24:11.917 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 33/156 2024-05-03T20:24:11.917 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 34/156 2024-05-03T20:24:11.917 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 35/156 2024-05-03T20:24:11.917 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 36/156 2024-05-03T20:24:11.917 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 37/156 2024-05-03T20:24:11.917 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 38/156 2024-05-03T20:24:11.917 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 39/156 2024-05-03T20:24:11.917 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-05-03T20:24:11.917 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-05-03T20:24:11.917 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-05-03T20:24:11.917 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-05-03T20:24:11.917 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-05-03T20:24:11.917 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-05-03T20:24:11.918 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-05-03T20:24:11.918 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-05-03T20:24:11.918 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-05-03T20:24:11.918 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-05-03T20:24:11.918 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-05-03T20:24:11.918 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-05-03T20:24:11.918 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-05-03T20:24:11.918 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-05-03T20:24:11.918 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-05-03T20:24:11.918 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-05-03T20:24:11.918 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-05-03T20:24:11.918 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-05-03T20:24:11.918 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-05-03T20:24:11.918 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-05-03T20:24:11.919 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-05-03T20:24:11.919 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-05-03T20:24:11.919 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-05-03T20:24:11.919 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-05-03T20:24:11.919 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-05-03T20:24:11.919 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-05-03T20:24:11.919 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-05-03T20:24:11.919 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-05-03T20:24:11.919 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-05-03T20:24:11.919 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-05-03T20:24:11.919 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-05-03T20:24:11.919 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-05-03T20:24:11.919 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-05-03T20:24:11.919 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-05-03T20:24:11.919 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-05-03T20:24:11.919 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-05-03T20:24:11.920 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-05-03T20:24:11.920 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-05-03T20:24:11.920 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-05-03T20:24:11.920 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-05-03T20:24:11.920 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-05-03T20:24:11.920 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-05-03T20:24:11.920 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-05-03T20:24:11.920 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-05-03T20:24:11.920 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-05-03T20:24:11.920 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-05-03T20:24:11.920 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-05-03T20:24:11.920 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-05-03T20:24:11.920 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-05-03T20:24:11.920 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-05-03T20:24:11.920 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-05-03T20:24:11.920 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-05-03T20:24:11.921 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-05-03T20:24:11.921 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-05-03T20:24:11.921 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-05-03T20:24:11.921 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-05-03T20:24:11.921 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-05-03T20:24:11.921 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-05-03T20:24:11.921 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-05-03T20:24:11.921 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-05-03T20:24:11.921 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-05-03T20:24:11.921 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-05-03T20:24:11.921 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-05-03T20:24:11.921 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-05-03T20:24:11.921 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-05-03T20:24:11.921 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-05-03T20:24:11.922 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-05-03T20:24:11.922 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-05-03T20:24:11.922 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-05-03T20:24:11.922 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-05-03T20:24:11.922 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-05-03T20:24:11.922 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-05-03T20:24:11.922 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-05-03T20:24:11.922 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-05-03T20:24:11.922 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-05-03T20:24:11.922 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-05-03T20:24:11.922 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-05-03T20:24:11.922 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-05-03T20:24:11.922 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T20:24:11.923 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T20:24:11.923 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-05-03T20:24:11.923 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-05-03T20:24:11.923 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-05-03T20:24:11.923 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-05-03T20:24:11.923 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-05-03T20:24:11.923 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-05-03T20:24:11.923 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-05-03T20:24:11.923 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-05-03T20:24:11.923 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-05-03T20:24:11.923 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-05-03T20:24:11.924 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-05-03T20:24:11.924 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-05-03T20:24:11.924 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-05-03T20:24:11.924 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-05-03T20:24:11.924 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-05-03T20:24:11.924 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-05-03T20:24:11.924 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-05-03T20:24:11.924 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-05-03T20:24:11.924 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-05-03T20:24:11.924 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-05-03T20:24:11.924 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-05-03T20:24:11.924 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-05-03T20:24:11.924 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-05-03T20:24:11.924 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-05-03T20:24:11.924 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-05-03T20:24:11.925 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-05-03T20:24:11.925 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-05-03T20:24:11.925 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-05-03T20:24:11.925 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-05-03T20:24:11.925 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 149/156 2024-05-03T20:24:11.925 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-05-03T20:24:11.925 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 151/156 2024-05-03T20:24:11.925 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-05-03T20:24:11.925 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-05-03T20:24:11.925 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T20:24:11.925 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-05-03T20:24:12.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T20:24:12.768 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-03T20:24:12.768 INFO:teuthology.orchestra.run.smithi204.stdout:Upgraded: 2024-05-03T20:24:12.768 INFO:teuthology.orchestra.run.smithi204.stdout: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.768 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.769 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-03T20:24:12.769 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-03T20:24:12.769 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-05-03T20:24:12.769 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-03T20:24:12.769 INFO:teuthology.orchestra.run.smithi204.stdout: automake-1.16.2-8.el9.noarch 2024-05-03T20:24:12.769 INFO:teuthology.orchestra.run.smithi204.stdout: bison-3.7.4-5.el9.x86_64 2024-05-03T20:24:12.769 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-03T20:24:12.769 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.769 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.769 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.769 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.769 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:12.770 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.770 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.770 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.770 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:12.770 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:12.770 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:12.770 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:12.770 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:12.770 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.770 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.770 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:12.770 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.771 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.771 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.771 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:12.771 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:12.771 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.771 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.771 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:12.771 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-03T20:24:12.771 INFO:teuthology.orchestra.run.smithi204.stdout: flex-2.6.4-9.el9.x86_64 2024-05-03T20:24:12.771 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-03T20:24:12.771 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-03T20:24:12.771 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-03T20:24:12.771 INFO:teuthology.orchestra.run.smithi204.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-03T20:24:12.772 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-03T20:24:12.772 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-03T20:24:12.772 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-03T20:24:12.772 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-03T20:24:12.772 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-03T20:24:12.772 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.772 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.772 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.772 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-03T20:24:12.772 INFO:teuthology.orchestra.run.smithi204.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-03T20:24:12.772 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-03T20:24:12.772 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-03T20:24:12.772 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-03T20:24:12.772 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.773 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.773 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-03T20:24:12.773 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.773 INFO:teuthology.orchestra.run.smithi204.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-03T20:24:12.773 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-03T20:24:12.773 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-03T20:24:12.774 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-03T20:24:12.774 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-03T20:24:12.774 INFO:teuthology.orchestra.run.smithi204.stdout: lua-5.4.4-4.el9.x86_64 2024-05-03T20:24:12.774 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-03T20:24:12.774 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-03T20:24:12.774 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-03T20:24:12.774 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-03T20:24:12.774 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-03T20:24:12.774 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-03T20:24:12.774 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-13.14-1.el9.x86_64 2024-05-03T20:24:12.774 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-03T20:24:12.774 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-03T20:24:12.775 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-03T20:24:12.775 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-03T20:24:12.775 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-03T20:24:12.775 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-03T20:24:12.775 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-03T20:24:12.775 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-03T20:24:12.775 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-03T20:24:12.775 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-03T20:24:12.775 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.775 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.775 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.775 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-03T20:24:12.775 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-03T20:24:12.775 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-03T20:24:12.776 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-03T20:24:12.776 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-03T20:24:12.776 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-03T20:24:12.776 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-03T20:24:12.776 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-03T20:24:12.776 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-03T20:24:12.776 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-03T20:24:12.776 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-03T20:24:12.776 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-03T20:24:12.776 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-03T20:24:12.776 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-03T20:24:12.776 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-03T20:24:12.776 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-03T20:24:12.776 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-03T20:24:12.776 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-03T20:24:12.777 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-03T20:24:12.777 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-03T20:24:12.777 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-03T20:24:12.777 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-03T20:24:12.777 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-03T20:24:12.777 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-03T20:24:12.777 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-03T20:24:12.777 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-03T20:24:12.777 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-03T20:24:12.777 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-03T20:24:12.777 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-03T20:24:12.777 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-03T20:24:12.777 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply-3.11-14.el9.noarch 2024-05-03T20:24:12.777 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-03T20:24:12.778 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-03T20:24:12.778 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-03T20:24:12.778 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-03T20:24:12.778 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-03T20:24:12.778 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-03T20:24:12.778 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-03T20:24:12.778 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-03T20:24:12.778 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-03T20:24:12.778 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.778 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.778 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-03T20:24:12.778 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-03T20:24:12.779 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-03T20:24:12.779 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.779 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-03T20:24:12.779 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-03T20:24:12.779 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-03T20:24:12.779 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-03T20:24:12.779 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-03T20:24:12.779 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-03T20:24:12.779 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-03T20:24:12.779 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-03T20:24:12.779 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-03T20:24:12.779 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-03T20:24:12.779 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-03T20:24:12.780 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-03T20:24:12.780 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-03T20:24:12.780 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-03T20:24:12.780 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-03T20:24:12.780 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.780 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.780 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:12.780 INFO:teuthology.orchestra.run.smithi204.stdout: re2-1:20211101-3.el9.x86_64 2024-05-03T20:24:12.780 INFO:teuthology.orchestra.run.smithi204.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-03T20:24:12.780 INFO:teuthology.orchestra.run.smithi204.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-03T20:24:12.780 INFO:teuthology.orchestra.run.smithi204.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-03T20:24:12.780 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-03T20:24:12.781 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-03T20:24:12.781 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-03T20:24:12.781 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-03T20:24:12.781 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-05-03T20:24:13.174 DEBUG:teuthology.parallel:result is None 2024-05-03T20:24:14.094 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T20:24:14.094 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:24:14.094 INFO:teuthology.orchestra.run.smithi181.stdout:Upgraded: 2024-05-03T20:24:14.094 INFO:teuthology.orchestra.run.smithi181.stdout: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.094 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.095 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-03T20:24:14.095 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-03T20:24:14.095 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2024-05-03T20:24:14.095 INFO:teuthology.orchestra.run.smithi181.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-03T20:24:14.095 INFO:teuthology.orchestra.run.smithi181.stdout: automake-1.16.2-8.el9.noarch 2024-05-03T20:24:14.095 INFO:teuthology.orchestra.run.smithi181.stdout: bison-3.7.4-5.el9.x86_64 2024-05-03T20:24:14.095 INFO:teuthology.orchestra.run.smithi181.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-03T20:24:14.095 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.095 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.095 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.095 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.095 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:14.095 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.095 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.096 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.096 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:14.096 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:14.096 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:14.096 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:14.096 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:14.096 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.096 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.096 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:14.096 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.096 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.096 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.096 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:14.096 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:14.097 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.097 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.097 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T20:24:14.097 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-03T20:24:14.097 INFO:teuthology.orchestra.run.smithi181.stdout: flex-2.6.4-9.el9.x86_64 2024-05-03T20:24:14.097 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-03T20:24:14.097 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-03T20:24:14.097 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-03T20:24:14.097 INFO:teuthology.orchestra.run.smithi181.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-03T20:24:14.097 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-03T20:24:14.097 INFO:teuthology.orchestra.run.smithi181.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-03T20:24:14.097 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-03T20:24:14.097 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-03T20:24:14.097 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-03T20:24:14.098 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.098 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.098 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.098 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-03T20:24:14.098 INFO:teuthology.orchestra.run.smithi181.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-03T20:24:14.098 INFO:teuthology.orchestra.run.smithi181.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-03T20:24:14.098 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-03T20:24:14.098 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-03T20:24:14.098 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.098 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.098 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-03T20:24:14.098 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.098 INFO:teuthology.orchestra.run.smithi181.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-03T20:24:14.098 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-03T20:24:14.099 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-03T20:24:14.099 INFO:teuthology.orchestra.run.smithi181.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-03T20:24:14.099 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-03T20:24:14.099 INFO:teuthology.orchestra.run.smithi181.stdout: lua-5.4.4-4.el9.x86_64 2024-05-03T20:24:14.099 INFO:teuthology.orchestra.run.smithi181.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-03T20:24:14.099 INFO:teuthology.orchestra.run.smithi181.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-03T20:24:14.099 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-03T20:24:14.099 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-03T20:24:14.099 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-03T20:24:14.099 INFO:teuthology.orchestra.run.smithi181.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-03T20:24:14.099 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-13.14-1.el9.x86_64 2024-05-03T20:24:14.099 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-03T20:24:14.099 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-03T20:24:14.099 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-03T20:24:14.100 INFO:teuthology.orchestra.run.smithi181.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-03T20:24:14.100 INFO:teuthology.orchestra.run.smithi181.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-03T20:24:14.100 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-03T20:24:14.100 INFO:teuthology.orchestra.run.smithi181.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-03T20:24:14.100 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-03T20:24:14.100 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-03T20:24:14.100 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-03T20:24:14.100 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.100 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.100 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.100 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-03T20:24:14.100 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-03T20:24:14.100 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-03T20:24:14.100 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-03T20:24:14.100 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-03T20:24:14.101 INFO:teuthology.orchestra.run.smithi181.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-03T20:24:14.101 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-03T20:24:14.101 INFO:teuthology.orchestra.run.smithi181.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-03T20:24:14.101 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-03T20:24:14.101 INFO:teuthology.orchestra.run.smithi181.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-03T20:24:14.101 INFO:teuthology.orchestra.run.smithi181.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-03T20:24:14.101 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-03T20:24:14.101 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-03T20:24:14.101 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-03T20:24:14.101 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-03T20:24:14.101 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-03T20:24:14.101 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-03T20:24:14.101 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-03T20:24:14.101 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-03T20:24:14.101 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-03T20:24:14.102 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-03T20:24:14.102 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-03T20:24:14.102 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-03T20:24:14.102 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-03T20:24:14.102 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-03T20:24:14.102 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-03T20:24:14.102 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-03T20:24:14.102 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-03T20:24:14.102 INFO:teuthology.orchestra.run.smithi181.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-03T20:24:14.102 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-03T20:24:14.102 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ply-3.11-14.el9.noarch 2024-05-03T20:24:14.102 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-03T20:24:14.102 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-03T20:24:14.102 INFO:teuthology.orchestra.run.smithi181.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-03T20:24:14.103 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-03T20:24:14.103 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-03T20:24:14.103 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-03T20:24:14.103 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-03T20:24:14.103 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-03T20:24:14.103 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-03T20:24:14.103 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.103 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.103 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-03T20:24:14.103 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-03T20:24:14.103 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-03T20:24:14.103 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.103 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-03T20:24:14.104 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-03T20:24:14.104 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-03T20:24:14.104 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-03T20:24:14.104 INFO:teuthology.orchestra.run.smithi181.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-03T20:24:14.104 INFO:teuthology.orchestra.run.smithi181.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-03T20:24:14.104 INFO:teuthology.orchestra.run.smithi181.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-03T20:24:14.104 INFO:teuthology.orchestra.run.smithi181.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-03T20:24:14.104 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-03T20:24:14.104 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-03T20:24:14.104 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-03T20:24:14.104 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-03T20:24:14.104 INFO:teuthology.orchestra.run.smithi181.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-03T20:24:14.104 INFO:teuthology.orchestra.run.smithi181.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-03T20:24:14.105 INFO:teuthology.orchestra.run.smithi181.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-03T20:24:14.105 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.105 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.105 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T20:24:14.105 INFO:teuthology.orchestra.run.smithi181.stdout: re2-1:20211101-3.el9.x86_64 2024-05-03T20:24:14.105 INFO:teuthology.orchestra.run.smithi181.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-03T20:24:14.105 INFO:teuthology.orchestra.run.smithi181.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-03T20:24:14.105 INFO:teuthology.orchestra.run.smithi181.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-03T20:24:14.105 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-03T20:24:14.105 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-03T20:24:14.105 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-03T20:24:14.106 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:24:14.106 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-05-03T20:24:14.481 DEBUG:teuthology.parallel:result is None 2024-05-03T20:24:14.481 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T20:24:14.621 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-03T20:24:14.642 INFO:teuthology.orchestra.run.smithi181.stdout:19.0.0-3450.g9a047fe7.el9 2024-05-03T20:24:14.643 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3450.g9a047fe7.el9 2024-05-03T20:24:14.643 INFO:teuthology.task.install:The correct ceph version 19.0.0-3450.g9a047fe7 is installed. 2024-05-03T20:24:14.645 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T20:24:14.778 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-03T20:24:14.803 INFO:teuthology.orchestra.run.smithi196.stdout:19.0.0-3450.g9a047fe7.el9 2024-05-03T20:24:14.804 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3450.g9a047fe7.el9 2024-05-03T20:24:14.804 INFO:teuthology.task.install:The correct ceph version 19.0.0-3450.g9a047fe7 is installed. 2024-05-03T20:24:14.806 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T20:24:15.015 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-03T20:24:15.036 INFO:teuthology.orchestra.run.smithi204.stdout:19.0.0-3450.g9a047fe7.el9 2024-05-03T20:24:15.036 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3450.g9a047fe7.el9 2024-05-03T20:24:15.036 INFO:teuthology.task.install:The correct ceph version 19.0.0-3450.g9a047fe7 is installed. 2024-05-03T20:24:15.038 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-03T20:24:15.038 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-03T20:24:15.038 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-03T20:24:15.064 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-03T20:24:15.064 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-03T20:24:15.094 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-03T20:24:15.094 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-03T20:24:15.120 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-03T20:24:15.120 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-03T20:24:15.120 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/daemon-helper 2024-05-03T20:24:15.143 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-03T20:24:15.207 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-03T20:24:15.207 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/daemon-helper 2024-05-03T20:24:15.235 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-03T20:24:15.303 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-03T20:24:15.303 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/daemon-helper 2024-05-03T20:24:15.327 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-03T20:24:15.390 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-03T20:24:15.391 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-03T20:24:15.391 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-03T20:24:15.413 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-03T20:24:15.477 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-03T20:24:15.478 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-03T20:24:15.506 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-03T20:24:15.574 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-03T20:24:15.574 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-03T20:24:15.597 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-03T20:24:15.660 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-03T20:24:15.661 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-03T20:24:15.661 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/stdin-killer 2024-05-03T20:24:15.684 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-03T20:24:15.750 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-03T20:24:15.750 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/stdin-killer 2024-05-03T20:24:15.777 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-03T20:24:15.845 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-03T20:24:15.845 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/stdin-killer 2024-05-03T20:24:15.868 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-03T20:24:15.932 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-03T20:24:16.028 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-05-03T20:24:16.028 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T20:24:16.029 INFO:tasks.cephadm:Cluster fsid is 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:24:16.029 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-03T20:24:16.029 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.181', 'mon.b': '172.21.15.196', 'mon.c': '172.21.15.204'} 2024-05-03T20:24:16.030 INFO:tasks.cephadm:First mon is mon.a on smithi181 2024-05-03T20:24:16.030 INFO:tasks.cephadm:First mgr is x 2024-05-03T20:24:16.030 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-03T20:24:16.030 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname $(hostname -s) 2024-05-03T20:24:16.055 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname $(hostname -s) 2024-05-03T20:24:16.083 DEBUG:teuthology.orchestra.run.smithi204:> sudo hostname $(hostname -s) 2024-05-03T20:24:16.107 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-03T20:24:16.107 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T20:24:16.318 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79034/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3450-g9a047fe7', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3450.g9a047fe7'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-02 00:36:37.664792', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240501.193033', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-03T20:24:16.453 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-rishabh-testing-20240501.193033, sha1 9a047fe7599a549042d06a3fb5b6813a203eabea from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T20:24:16.454 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/x86_64/flavors/default/cephadm 2024-05-03T20:24:16.454 DEBUG:teuthology.orchestra.run.smithi181:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-03T20:24:16.918 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r--. 1 ubuntu ubuntu 782467 May 3 20:24 /home/ubuntu/cephtest/cephadm 2024-05-03T20:24:16.918 DEBUG:teuthology.orchestra.run.smithi196:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-03T20:24:18.135 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r--. 1 ubuntu ubuntu 782467 May 3 20:24 /home/ubuntu/cephtest/cephadm 2024-05-03T20:24:18.135 DEBUG:teuthology.orchestra.run.smithi204:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-03T20:24:19.447 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r--. 1 ubuntu ubuntu 782467 May 3 20:24 /home/ubuntu/cephtest/cephadm 2024-05-03T20:24:19.448 DEBUG:teuthology.orchestra.run.smithi181:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-03T20:24:19.467 DEBUG:teuthology.orchestra.run.smithi196:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-03T20:24:19.486 DEBUG:teuthology.orchestra.run.smithi204:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-03T20:24:19.520 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea on all hosts... 2024-05-03T20:24:19.520 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea pull 2024-05-03T20:24:19.523 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea pull 2024-05-03T20:24:19.529 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea pull 2024-05-03T20:24:21.096 INFO:teuthology.orchestra.run.smithi196.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea... 2024-05-03T20:24:21.124 INFO:teuthology.orchestra.run.smithi204.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea... 2024-05-03T20:24:21.250 INFO:teuthology.orchestra.run.smithi181.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea... 2024-05-03T20:24:56.973 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-05-03T20:24:56.973 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev)", 2024-05-03T20:24:56.973 INFO:teuthology.orchestra.run.smithi181.stdout: "image_id": "ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012", 2024-05-03T20:24:56.973 INFO:teuthology.orchestra.run.smithi181.stdout: "repo_digests": [ 2024-05-03T20:24:56.973 INFO:teuthology.orchestra.run.smithi181.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd" 2024-05-03T20:24:56.973 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-05-03T20:24:56.973 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-05-03T20:25:19.689 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-05-03T20:25:19.689 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev)", 2024-05-03T20:25:19.689 INFO:teuthology.orchestra.run.smithi196.stdout: "image_id": "ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012", 2024-05-03T20:25:19.689 INFO:teuthology.orchestra.run.smithi196.stdout: "repo_digests": [ 2024-05-03T20:25:19.689 INFO:teuthology.orchestra.run.smithi196.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd" 2024-05-03T20:25:19.689 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-05-03T20:25:19.689 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-05-03T20:25:20.109 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-05-03T20:25:20.109 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_version": "ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev)", 2024-05-03T20:25:20.109 INFO:teuthology.orchestra.run.smithi204.stdout: "image_id": "ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012", 2024-05-03T20:25:20.109 INFO:teuthology.orchestra.run.smithi204.stdout: "repo_digests": [ 2024-05-03T20:25:20.109 INFO:teuthology.orchestra.run.smithi204.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd" 2024-05-03T20:25:20.109 INFO:teuthology.orchestra.run.smithi204.stdout: ] 2024-05-03T20:25:20.110 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-05-03T20:25:20.134 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /etc/ceph 2024-05-03T20:25:20.167 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph 2024-05-03T20:25:20.197 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /etc/ceph 2024-05-03T20:25:20.223 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 777 /etc/ceph 2024-05-03T20:25:20.250 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 777 /etc/ceph 2024-05-03T20:25:20.275 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 777 /etc/ceph 2024-05-03T20:25:20.299 INFO:tasks.cephadm:Writing seed config... 2024-05-03T20:25:20.301 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-03T20:25:20.301 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-03T20:25:20.301 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-03T20:25:20.301 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-03T20:25:20.301 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-03T20:25:20.301 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-03T20:25:20.301 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-03T20:25:20.302 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-03T20:25:20.302 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-03T20:25:20.302 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-03T20:25:20.302 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-03T20:25:20.302 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-03T20:25:20.302 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-03T20:25:20.302 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-03T20:25:20.302 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-03T20:25:20.302 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-03T20:25:20.302 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-03T20:25:20.302 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-05-03T20:25:20.302 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-03T20:25:20.302 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-03T20:25:20.302 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-03T20:25:20.303 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-03T20:25:20.303 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-03T20:25:20.303 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-03T20:25:20.303 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-03T20:25:20.303 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-03T20:25:20.303 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-03T20:25:20.303 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-03T20:25:20.303 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-03T20:25:20.303 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-03T20:25:20.303 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-05-03T20:25:20.303 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-05-03T20:25:20.303 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-05-03T20:25:20.303 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-03T20:25:20.303 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-03T20:25:20.303 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-05-03T20:25:20.304 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-05-03T20:25:20.304 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-03T20:25:20.304 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-03T20:25:20.304 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-05-03T20:25:20.304 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-03T20:25:20.304 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-03T20:25:20.304 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-03T20:25:20.304 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-03T20:25:20.304 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-03T20:25:20.304 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-03T20:25:20.304 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-03T20:25:20.304 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-03T20:25:20.305 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-03T20:25:20.305 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-03T20:25:20.322 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 1cd058ee-098b-11ef-bc97-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-03T20:25:20.322 DEBUG:teuthology.orchestra.run.smithi181:mon.a> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mon.a.service 2024-05-03T20:25:20.364 DEBUG:teuthology.orchestra.run.smithi181:mgr.x> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mgr.x.service 2024-05-03T20:25:20.407 INFO:tasks.cephadm:Bootstrapping... 2024-05-03T20:25:20.407 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea -v bootstrap --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.181 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-03T20:25:20.600 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-05-03T20:25:20.600 INFO:teuthology.orchestra.run.smithi181.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea', '-v', 'bootstrap', '--fsid', '1cd058ee-098b-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.181', '--skip-admin-label'] 2024-05-03T20:25:20.600 INFO:teuthology.orchestra.run.smithi181.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-03T20:25:20.600 INFO:teuthology.orchestra.run.smithi181.stdout:Verifying podman|docker is present... 2024-05-03T20:25:20.626 INFO:teuthology.orchestra.run.smithi181.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-03T20:25:20.626 INFO:teuthology.orchestra.run.smithi181.stdout:Verifying lvm2 is present... 2024-05-03T20:25:20.626 INFO:teuthology.orchestra.run.smithi181.stdout:Verifying time synchronization is in place... 2024-05-03T20:25:20.634 INFO:teuthology.orchestra.run.smithi181.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-03T20:25:20.634 INFO:teuthology.orchestra.run.smithi181.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-03T20:25:20.639 INFO:teuthology.orchestra.run.smithi181.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-03T20:25:20.640 INFO:teuthology.orchestra.run.smithi181.stdout:systemctl: stdout inactive 2024-05-03T20:25:20.646 INFO:teuthology.orchestra.run.smithi181.stdout:systemctl: stdout enabled 2024-05-03T20:25:20.652 INFO:teuthology.orchestra.run.smithi181.stdout:systemctl: stdout active 2024-05-03T20:25:20.652 INFO:teuthology.orchestra.run.smithi181.stdout:Unit chronyd.service is enabled and running 2024-05-03T20:25:20.652 INFO:teuthology.orchestra.run.smithi181.stdout:Repeating the final host check... 2024-05-03T20:25:20.676 INFO:teuthology.orchestra.run.smithi181.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-03T20:25:20.676 INFO:teuthology.orchestra.run.smithi181.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-03T20:25:20.677 INFO:teuthology.orchestra.run.smithi181.stdout:systemctl is present 2024-05-03T20:25:20.677 INFO:teuthology.orchestra.run.smithi181.stdout:lvcreate is present 2024-05-03T20:25:20.683 INFO:teuthology.orchestra.run.smithi181.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-03T20:25:20.683 INFO:teuthology.orchestra.run.smithi181.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-03T20:25:20.689 INFO:teuthology.orchestra.run.smithi181.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-03T20:25:20.689 INFO:teuthology.orchestra.run.smithi181.stdout:systemctl: stdout inactive 2024-05-03T20:25:20.695 INFO:teuthology.orchestra.run.smithi181.stdout:systemctl: stdout enabled 2024-05-03T20:25:20.701 INFO:teuthology.orchestra.run.smithi181.stdout:systemctl: stdout active 2024-05-03T20:25:20.701 INFO:teuthology.orchestra.run.smithi181.stdout:Unit chronyd.service is enabled and running 2024-05-03T20:25:20.701 INFO:teuthology.orchestra.run.smithi181.stdout:Host looks OK 2024-05-03T20:25:20.701 INFO:teuthology.orchestra.run.smithi181.stdout:Cluster fsid: 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:25:20.701 INFO:teuthology.orchestra.run.smithi181.stdout:Acquiring lock 140636268903728 on /run/cephadm/1cd058ee-098b-11ef-bc97-c7b262605968.lock 2024-05-03T20:25:20.702 INFO:teuthology.orchestra.run.smithi181.stdout:Lock 140636268903728 acquired on /run/cephadm/1cd058ee-098b-11ef-bc97-c7b262605968.lock 2024-05-03T20:25:20.702 INFO:teuthology.orchestra.run.smithi181.stdout:Verifying IP 172.21.15.181 port 3300 ... 2024-05-03T20:25:20.702 INFO:teuthology.orchestra.run.smithi181.stdout:Verifying IP 172.21.15.181 port 6789 ... 2024-05-03T20:25:20.702 INFO:teuthology.orchestra.run.smithi181.stdout:Base mon IP(s) is [172.21.15.181:3300, 172.21.15.181:6789], mon addrv is [v2:172.21.15.181:3300,v1:172.21.15.181:6789] 2024-05-03T20:25:20.705 INFO:teuthology.orchestra.run.smithi181.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.181 metric 100 2024-05-03T20:25:20.705 INFO:teuthology.orchestra.run.smithi181.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.181 metric 100 2024-05-03T20:25:20.708 INFO:teuthology.orchestra.run.smithi181.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-03T20:25:20.708 INFO:teuthology.orchestra.run.smithi181.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-05-03T20:25:20.708 INFO:teuthology.orchestra.run.smithi181.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-05-03T20:25:20.710 INFO:teuthology.orchestra.run.smithi181.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-03T20:25:20.710 INFO:teuthology.orchestra.run.smithi181.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-03T20:25:20.710 INFO:teuthology.orchestra.run.smithi181.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-03T20:25:20.710 INFO:teuthology.orchestra.run.smithi181.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-03T20:25:20.710 INFO:teuthology.orchestra.run.smithi181.stdout:/sbin/ip: stdout inet6 fe80::8c6:cae9:edfa:417a/64 scope link noprefixroute 2024-05-03T20:25:20.710 INFO:teuthology.orchestra.run.smithi181.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-03T20:25:20.711 INFO:teuthology.orchestra.run.smithi181.stdout:Mon IP `172.21.15.181` is in CIDR network `172.21.0.0/20` 2024-05-03T20:25:20.711 INFO:teuthology.orchestra.run.smithi181.stdout:Mon IP `172.21.15.181` is in CIDR network `172.21.0.0/20` 2024-05-03T20:25:20.711 INFO:teuthology.orchestra.run.smithi181.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-03T20:25:20.711 INFO:teuthology.orchestra.run.smithi181.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-03T20:25:20.712 INFO:teuthology.orchestra.run.smithi181.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea... 2024-05-03T20:25:21.129 INFO:teuthology.orchestra.run.smithi181.stdout:/bin/podman: stdout ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 2024-05-03T20:25:21.129 INFO:teuthology.orchestra.run.smithi181.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea... 2024-05-03T20:25:21.129 INFO:teuthology.orchestra.run.smithi181.stdout:/bin/podman: stderr Getting image source signatures 2024-05-03T20:25:21.129 INFO:teuthology.orchestra.run.smithi181.stdout:/bin/podman: stderr Copying blob sha256:8173fd91c96404ae0424bb08037c57749051dcbdfaeee9354a65e201dad7b0e2 2024-05-03T20:25:21.130 INFO:teuthology.orchestra.run.smithi181.stdout:/bin/podman: stderr Copying blob sha256:e2f2fd2adca92139da4b4b907fa90a6775d3eb4eddaa977c42ef37c66eea70bd 2024-05-03T20:25:21.130 INFO:teuthology.orchestra.run.smithi181.stdout:/bin/podman: stderr Copying config sha256:ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 2024-05-03T20:25:21.130 INFO:teuthology.orchestra.run.smithi181.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-03T20:25:23.299 INFO:teuthology.orchestra.run.smithi181.stdout:ceph: stdout ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev) 2024-05-03T20:25:23.299 INFO:teuthology.orchestra.run.smithi181.stdout:Ceph version: ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev) 2024-05-03T20:25:23.299 INFO:teuthology.orchestra.run.smithi181.stdout:Extracting ceph user uid/gid from container image... 2024-05-03T20:25:32.916 INFO:teuthology.orchestra.run.smithi181.stdout:stat: stdout 167 167 2024-05-03T20:25:32.917 INFO:teuthology.orchestra.run.smithi181.stdout:Creating initial keys... 2024-05-03T20:25:34.300 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph-authtool: stdout AQA9SDVm8ssoKxAAU4f20Kb/Fm2GQI3WRXNcsA== 2024-05-03T20:25:35.281 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph-authtool: stdout AQA+SDVmFk1kJxAACGw9KN1DVC3xnjet3yLmjg== 2024-05-03T20:25:36.170 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph-authtool: stdout AQA/SDVmzrG1JBAA2ICbKcEMQDHi4BH5i7NY4w== 2024-05-03T20:25:36.171 INFO:teuthology.orchestra.run.smithi181.stdout:Creating initial monmap... 2024-05-03T20:25:37.085 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-03T20:25:37.085 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-03T20:25:37.085 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:25:37.085 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-03T20:25:37.085 INFO:teuthology.orchestra.run.smithi181.stdout:monmaptool for a [v2:172.21.15.181:3300,v1:172.21.15.181:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-03T20:25:37.085 INFO:teuthology.orchestra.run.smithi181.stdout:setting min_mon_release = quincy 2024-05-03T20:25:37.085 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/monmaptool: set fsid to 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:25:37.086 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-03T20:25:37.086 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:25:37.086 INFO:teuthology.orchestra.run.smithi181.stdout:Creating mon... 2024-05-03T20:25:37.965 INFO:teuthology.orchestra.run.smithi181.stdout:create mon.a on 2024-05-03T20:25:38.162 INFO:teuthology.orchestra.run.smithi181.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-03T20:25:38.297 INFO:teuthology.orchestra.run.smithi181.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-03T20:25:38.502 INFO:teuthology.orchestra.run.smithi181.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-1cd058ee-098b-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-1cd058ee-098b-11ef-bc97-c7b262605968.target. 2024-05-03T20:25:38.502 INFO:teuthology.orchestra.run.smithi181.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-1cd058ee-098b-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-1cd058ee-098b-11ef-bc97-c7b262605968.target. 2024-05-03T20:25:38.742 INFO:teuthology.orchestra.run.smithi181.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mon.a 2024-05-03T20:25:38.742 INFO:teuthology.orchestra.run.smithi181.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mon.a.service: Unit ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mon.a.service not loaded. 2024-05-03T20:25:38.902 INFO:teuthology.orchestra.run.smithi181.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1cd058ee-098b-11ef-bc97-c7b262605968.target.wants/ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mon.a.service → /etc/systemd/system/ceph-1cd058ee-098b-11ef-bc97-c7b262605968@.service. 2024-05-03T20:25:39.168 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:39 smithi181 podman[29412]: 2024-05-03 20:25:39.071218909 +0000 UTC m=+0.018169329 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T20:25:39.431 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:39 smithi181 podman[29412]: 2024-05-03 20:25:39.385427363 +0000 UTC m=+0.332377765 container init a5df4e249393287f2eb5b1e16e4d1910359c5511159f5af05ba822a885a42a15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-a, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T20:25:39.431 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:39 smithi181 podman[29412]: 2024-05-03 20:25:39.388648341 +0000 UTC m=+0.335598743 container start a5df4e249393287f2eb5b1e16e4d1910359c5511159f5af05ba822a885a42a15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-a, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux ) 2024-05-03T20:25:39.473 INFO:teuthology.orchestra.run.smithi181.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-03T20:25:39.473 INFO:teuthology.orchestra.run.smithi181.stdout:systemctl: stdout disabled 2024-05-03T20:25:39.480 INFO:teuthology.orchestra.run.smithi181.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-03T20:25:39.480 INFO:teuthology.orchestra.run.smithi181.stdout:systemctl: stdout inactive 2024-05-03T20:25:39.480 INFO:teuthology.orchestra.run.smithi181.stdout:firewalld.service is not enabled 2024-05-03T20:25:39.480 INFO:teuthology.orchestra.run.smithi181.stdout:Not possible to enable service . firewalld.service is not available 2024-05-03T20:25:39.481 INFO:teuthology.orchestra.run.smithi181.stdout:Waiting for mon to start... 2024-05-03T20:25:39.481 INFO:teuthology.orchestra.run.smithi181.stdout:Waiting for mon... 2024-05-03T20:25:39.694 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:39 smithi181 ceph-mon[29431]: mkfs 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:25:39.695 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:39 smithi181 ceph-mon[29431]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-03T20:25:39.695 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:39 smithi181 bash[29412]: a5df4e249393287f2eb5b1e16e4d1910359c5511159f5af05ba822a885a42a15 2024-05-03T20:25:39.695 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:39 smithi181 systemd[1]: Started Ceph mon.a for 1cd058ee-098b-11ef-bc97-c7b262605968. 2024-05-03T20:25:40.541 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:40 smithi181 ceph-mon[29431]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-03T20:25:40.541 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:40 smithi181 ceph-mon[29431]: monmap epoch 1 2024-05-03T20:25:40.541 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:40 smithi181 ceph-mon[29431]: fsid 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:25:40.542 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:40 smithi181 ceph-mon[29431]: last_changed 2024-05-03T20:25:36.511893+0000 2024-05-03T20:25:40.542 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:40 smithi181 ceph-mon[29431]: created 2024-05-03T20:25:36.511893+0000 2024-05-03T20:25:40.542 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:40 smithi181 ceph-mon[29431]: min_mon_release 19 (squid) 2024-05-03T20:25:40.542 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:40 smithi181 ceph-mon[29431]: election_strategy: 1 2024-05-03T20:25:40.542 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:40 smithi181 ceph-mon[29431]: 0: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.a 2024-05-03T20:25:40.542 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:40 smithi181 ceph-mon[29431]: fsmap 2024-05-03T20:25:40.542 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:40 smithi181 ceph-mon[29431]: osdmap e1: 0 total, 0 up, 0 in 2024-05-03T20:25:40.542 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:40 smithi181 ceph-mon[29431]: mgrmap e1: no daemons active 2024-05-03T20:25:40.542 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:40 smithi181 ceph-mon[29431]: from='client.? 172.21.15.181:0/1226488865' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-03T20:25:40.549 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout cluster: 2024-05-03T20:25:40.549 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout id: 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:25:40.550 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-03T20:25:40.550 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout 2024-05-03T20:25:40.550 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout services: 2024-05-03T20:25:40.550 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.501652s) 2024-05-03T20:25:40.550 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-03T20:25:40.550 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-03T20:25:40.550 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout 2024-05-03T20:25:40.550 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout data: 2024-05-03T20:25:40.550 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-03T20:25:40.550 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-03T20:25:40.550 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-03T20:25:40.550 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout pgs: 2024-05-03T20:25:40.550 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout 2024-05-03T20:25:40.550 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.852+0000 7fceed3d1700 1 Processor -- start 2024-05-03T20:25:40.551 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.852+0000 7fceed3d1700 1 -- start start 2024-05-03T20:25:40.551 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.852+0000 7fceed3d1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee8109a60 0x7fcee810be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:25:40.551 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.852+0000 7fceed3d1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcee807ce50 con 0x7fcee8109a60 2024-05-03T20:25:40.551 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.852+0000 7fcee6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee8109a60 0x7fcee810be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:25:40.551 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.852+0000 7fcee6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee8109a60 0x7fcee810be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35220/0 (socket says 172.21.15.181:35220) 2024-05-03T20:25:40.551 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.852+0000 7fcee6ffd700 1 -- 172.21.15.181:0/4093350621 learned_addr learned my addr 172.21.15.181:0/4093350621 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:25:40.551 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.853+0000 7fcee6ffd700 1 -- 172.21.15.181:0/4093350621 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee810c390 con 0x7fcee8109a60 2024-05-03T20:25:40.551 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.853+0000 7fcee6ffd700 1 --2- 172.21.15.181:0/4093350621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee8109a60 0x7fcee810be50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fced80098a0 tx=0x7fced8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4d1d39ce4283d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:25:40.551 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.853+0000 7fcee5ffb700 1 -- 172.21.15.181:0/4093350621 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fced8004160 con 0x7fcee8109a60 2024-05-03T20:25:40.551 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.853+0000 7fcee5ffb700 1 -- 172.21.15.181:0/4093350621 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fced8004300 con 0x7fcee8109a60 2024-05-03T20:25:40.551 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.853+0000 7fcee5ffb700 1 -- 172.21.15.181:0/4093350621 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fced80036a0 con 0x7fcee8109a60 2024-05-03T20:25:40.551 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.853+0000 7fceed3d1700 1 -- 172.21.15.181:0/4093350621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee8109a60 msgr2=0x7fcee810be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:25:40.551 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.853+0000 7fceed3d1700 1 --2- 172.21.15.181:0/4093350621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee8109a60 0x7fcee810be50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fced80098a0 tx=0x7fced8009bb0 comp rx=0 tx=0).stop 2024-05-03T20:25:40.551 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.853+0000 7fceed3d1700 1 -- 172.21.15.181:0/4093350621 shutdown_connections 2024-05-03T20:25:40.551 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.853+0000 7fceed3d1700 1 --2- 172.21.15.181:0/4093350621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee8109a60 0x7fcee810be50 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:25:40.552 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.853+0000 7fceed3d1700 1 -- 172.21.15.181:0/4093350621 >> 172.21.15.181:0/4093350621 conn(0x7fcee8103490 msgr2=0x7fcee81058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:25:40.552 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.853+0000 7fceed3d1700 1 -- 172.21.15.181:0/4093350621 shutdown_connections 2024-05-03T20:25:40.552 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.853+0000 7fceed3d1700 1 -- 172.21.15.181:0/4093350621 wait complete. 2024-05-03T20:25:40.552 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.854+0000 7fceed3d1700 1 Processor -- start 2024-05-03T20:25:40.552 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.854+0000 7fceed3d1700 1 -- start start 2024-05-03T20:25:40.552 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.854+0000 7fceed3d1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee8109a60 0x7fcee81092e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:25:40.552 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.854+0000 7fceed3d1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcee810c800 con 0x7fcee8109a60 2024-05-03T20:25:40.552 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.854+0000 7fcee6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee8109a60 0x7fcee81092e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:25:40.552 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.854+0000 7fcee6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee8109a60 0x7fcee81092e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35234/0 (socket says 172.21.15.181:35234) 2024-05-03T20:25:40.552 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.854+0000 7fcee6ffd700 1 -- 172.21.15.181:0/1226488865 learned_addr learned my addr 172.21.15.181:0/1226488865 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:25:40.552 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.855+0000 7fcee6ffd700 1 -- 172.21.15.181:0/1226488865 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee8109820 con 0x7fcee8109a60 2024-05-03T20:25:40.552 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.855+0000 7fcee6ffd700 1 --2- 172.21.15.181:0/1226488865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee8109a60 0x7fcee81092e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fced8000c00 tx=0x7fced8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:25:40.552 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.855+0000 7fcecffff700 1 -- 172.21.15.181:0/1226488865 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fced8003db0 con 0x7fcee8109a60 2024-05-03T20:25:40.552 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.855+0000 7fcecffff700 1 -- 172.21.15.181:0/1226488865 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fced8011630 con 0x7fcee8109a60 2024-05-03T20:25:40.553 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.855+0000 7fceed3d1700 1 -- 172.21.15.181:0/1226488865 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee8107a10 con 0x7fcee8109a60 2024-05-03T20:25:40.553 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.855+0000 7fcecffff700 1 -- 172.21.15.181:0/1226488865 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fced8011950 con 0x7fcee8109a60 2024-05-03T20:25:40.553 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.855+0000 7fceed3d1700 1 -- 172.21.15.181:0/1226488865 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee8107f50 con 0x7fcee8109a60 2024-05-03T20:25:40.553 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.855+0000 7fcecffff700 1 -- 172.21.15.181:0/1226488865 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fced801a460 con 0x7fcee8109a60 2024-05-03T20:25:40.553 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.855+0000 7fcecffff700 1 -- 172.21.15.181:0/1226488865 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fced801b780 con 0x7fcee8109a60 2024-05-03T20:25:40.553 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.856+0000 7fceed3d1700 1 -- 172.21.15.181:0/1226488865 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcee8064b00 con 0x7fcee8109a60 2024-05-03T20:25:40.553 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.858+0000 7fcecffff700 1 -- 172.21.15.181:0/1226488865 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fced801ba20 con 0x7fcee8109a60 2024-05-03T20:25:40.553 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.928+0000 7fceed3d1700 1 -- 172.21.15.181:0/1226488865 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fcee81960c0 con 0x7fcee8109a60 2024-05-03T20:25:40.553 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.928+0000 7fcecffff700 1 -- 172.21.15.181:0/1226488865 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fced801bc00 con 0x7fcee8109a60 2024-05-03T20:25:40.553 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.929+0000 7fceed3d1700 1 -- 172.21.15.181:0/1226488865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee8109a60 msgr2=0x7fcee81092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:25:40.553 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.929+0000 7fceed3d1700 1 --2- 172.21.15.181:0/1226488865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee8109a60 0x7fcee81092e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fced8000c00 tx=0x7fced8004750 comp rx=0 tx=0).stop 2024-05-03T20:25:40.553 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.929+0000 7fceed3d1700 1 -- 172.21.15.181:0/1226488865 shutdown_connections 2024-05-03T20:25:40.553 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.929+0000 7fceed3d1700 1 --2- 172.21.15.181:0/1226488865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee8109a60 0x7fcee81092e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:25:40.553 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.929+0000 7fceed3d1700 1 -- 172.21.15.181:0/1226488865 >> 172.21.15.181:0/1226488865 conn(0x7fcee8103490 msgr2=0x7fcee81042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:25:40.553 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.929+0000 7fceed3d1700 1 -- 172.21.15.181:0/1226488865 shutdown_connections 2024-05-03T20:25:40.553 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:39.929+0000 7fceed3d1700 1 -- 172.21.15.181:0/1226488865 wait complete. 2024-05-03T20:25:40.554 INFO:teuthology.orchestra.run.smithi181.stdout:mon is available 2024-05-03T20:25:40.554 INFO:teuthology.orchestra.run.smithi181.stdout:Assimilating anything we can from ceph.conf... 2024-05-03T20:25:41.614 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:41 smithi181 ceph-mon[29431]: from='client.? 172.21.15.181:0/1548498164' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-03T20:25:41.614 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:41 smithi181 ceph-mon[29431]: from='client.? 172.21.15.181:0/1548498164' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-03T20:25:41.624 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout 2024-05-03T20:25:41.624 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout [global] 2024-05-03T20:25:41.624 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout fsid = 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:25:41.624 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-03T20:25:41.624 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.181:3300,v1:172.21.15.181:6789] 2024-05-03T20:25:41.625 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-03T20:25:41.625 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-03T20:25:41.625 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-03T20:25:41.625 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-03T20:25:41.625 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout 2024-05-03T20:25:41.625 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-03T20:25:41.625 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-03T20:25:41.625 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout 2024-05-03T20:25:41.625 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout [osd] 2024-05-03T20:25:41.625 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-05-03T20:25:41.625 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-03T20:25:41.625 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-03T20:25:41.626 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-03T20:25:41.626 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-03T20:25:41.626 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-03T20:25:41.626 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.011+0000 7fd97c7c9700 1 Processor -- start 2024-05-03T20:25:41.626 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.011+0000 7fd97c7c9700 1 -- start start 2024-05-03T20:25:41.627 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.011+0000 7fd97c7c9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd97407eea0 0x7fd97407d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:25:41.627 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.011+0000 7fd97c7c9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd97407d8d0 con 0x7fd97407eea0 2024-05-03T20:25:41.627 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.012+0000 7fd97a565700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd97407eea0 0x7fd97407d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:25:41.627 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.012+0000 7fd97a565700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd97407eea0 0x7fd97407d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35238/0 (socket says 172.21.15.181:35238) 2024-05-03T20:25:41.627 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.012+0000 7fd97a565700 1 -- 172.21.15.181:0/3980698077 learned_addr learned my addr 172.21.15.181:0/3980698077 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:25:41.627 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.012+0000 7fd97a565700 1 -- 172.21.15.181:0/3980698077 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97407da50 con 0x7fd97407eea0 2024-05-03T20:25:41.628 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.012+0000 7fd97a565700 1 --2- 172.21.15.181:0/3980698077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd97407eea0 0x7fd97407d300 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd9700098a0 tx=0x7fd970009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde06634d72e0dfd server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:25:41.628 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.012+0000 7fd979563700 1 -- 172.21.15.181:0/3980698077 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd970004160 con 0x7fd97407eea0 2024-05-03T20:25:41.628 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.012+0000 7fd979563700 1 -- 172.21.15.181:0/3980698077 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd970004300 con 0x7fd97407eea0 2024-05-03T20:25:41.628 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.012+0000 7fd979563700 1 -- 172.21.15.181:0/3980698077 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9700036a0 con 0x7fd97407eea0 2024-05-03T20:25:41.628 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.013+0000 7fd97c7c9700 1 -- 172.21.15.181:0/3980698077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd97407eea0 msgr2=0x7fd97407d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:25:41.628 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.013+0000 7fd97c7c9700 1 --2- 172.21.15.181:0/3980698077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd97407eea0 0x7fd97407d300 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd9700098a0 tx=0x7fd970009bb0 comp rx=0 tx=0).stop 2024-05-03T20:25:41.628 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.013+0000 7fd97c7c9700 1 -- 172.21.15.181:0/3980698077 shutdown_connections 2024-05-03T20:25:41.628 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.013+0000 7fd97c7c9700 1 --2- 172.21.15.181:0/3980698077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd97407eea0 0x7fd97407d300 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:25:41.629 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.013+0000 7fd97c7c9700 1 -- 172.21.15.181:0/3980698077 >> 172.21.15.181:0/3980698077 conn(0x7fd974104510 msgr2=0x7fd974106930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:25:41.629 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.013+0000 7fd97c7c9700 1 -- 172.21.15.181:0/3980698077 shutdown_connections 2024-05-03T20:25:41.629 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.013+0000 7fd97c7c9700 1 -- 172.21.15.181:0/3980698077 wait complete. 2024-05-03T20:25:41.629 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.013+0000 7fd97c7c9700 1 Processor -- start 2024-05-03T20:25:41.629 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.013+0000 7fd97c7c9700 1 -- start start 2024-05-03T20:25:41.629 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.013+0000 7fd97c7c9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd97407eea0 0x7fd97419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:25:41.629 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.013+0000 7fd97c7c9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd97407df80 con 0x7fd97407eea0 2024-05-03T20:25:41.629 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.014+0000 7fd97a565700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd97407eea0 0x7fd97419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:25:41.629 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.014+0000 7fd97a565700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd97407eea0 0x7fd97419e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35244/0 (socket says 172.21.15.181:35244) 2024-05-03T20:25:41.629 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.014+0000 7fd97a565700 1 -- 172.21.15.181:0/1548498164 learned_addr learned my addr 172.21.15.181:0/1548498164 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:25:41.629 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.014+0000 7fd97a565700 1 -- 172.21.15.181:0/1548498164 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97419ec50 con 0x7fd97407eea0 2024-05-03T20:25:41.629 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.014+0000 7fd97a565700 1 --2- 172.21.15.181:0/1548498164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd97407eea0 0x7fd97419e710 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd970000c00 tx=0x7fd970004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:25:41.629 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.014+0000 7fd96b7fe700 1 -- 172.21.15.181:0/1548498164 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd970003db0 con 0x7fd97407eea0 2024-05-03T20:25:41.630 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.014+0000 7fd96b7fe700 1 -- 172.21.15.181:0/1548498164 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd970011630 con 0x7fd97407eea0 2024-05-03T20:25:41.630 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.014+0000 7fd97c7c9700 1 -- 172.21.15.181:0/1548498164 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd97419eee0 con 0x7fd97407eea0 2024-05-03T20:25:41.630 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.014+0000 7fd96b7fe700 1 -- 172.21.15.181:0/1548498164 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd970011950 con 0x7fd97407eea0 2024-05-03T20:25:41.630 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.014+0000 7fd97c7c9700 1 -- 172.21.15.181:0/1548498164 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9741a1bd0 con 0x7fd97407eea0 2024-05-03T20:25:41.630 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.014+0000 7fd96b7fe700 1 -- 172.21.15.181:0/1548498164 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd970011af0 con 0x7fd97407eea0 2024-05-03T20:25:41.631 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.014+0000 7fd96b7fe700 1 -- 172.21.15.181:0/1548498164 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd97001c450 con 0x7fd97407eea0 2024-05-03T20:25:41.631 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.015+0000 7fd97c7c9700 1 -- 172.21.15.181:0/1548498164 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd974064b00 con 0x7fd97407eea0 2024-05-03T20:25:41.631 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.017+0000 7fd96b7fe700 1 -- 172.21.15.181:0/1548498164 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd970032020 con 0x7fd97407eea0 2024-05-03T20:25:41.631 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.084+0000 7fd97c7c9700 1 -- 172.21.15.181:0/1548498164 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fd9741a1da0 con 0x7fd97407eea0 2024-05-03T20:25:41.631 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.088+0000 7fd96b7fe700 1 -- 172.21.15.181:0/1548498164 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+494 (secure 0 0 0) 0x7fd970004260 con 0x7fd97407eea0 2024-05-03T20:25:41.631 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.088+0000 7fd96b7fe700 1 -- 172.21.15.181:0/1548498164 <== mon.0 v2:172.21.15.181:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd97001c650 con 0x7fd97407eea0 2024-05-03T20:25:41.631 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.089+0000 7fd97c7c9700 1 -- 172.21.15.181:0/1548498164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd97407eea0 msgr2=0x7fd97419e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:25:41.631 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.089+0000 7fd97c7c9700 1 --2- 172.21.15.181:0/1548498164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd97407eea0 0x7fd97419e710 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd970000c00 tx=0x7fd970004750 comp rx=0 tx=0).stop 2024-05-03T20:25:41.631 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.089+0000 7fd97c7c9700 1 -- 172.21.15.181:0/1548498164 shutdown_connections 2024-05-03T20:25:41.631 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.089+0000 7fd97c7c9700 1 --2- 172.21.15.181:0/1548498164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd97407eea0 0x7fd97419e710 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:25:41.631 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.089+0000 7fd97c7c9700 1 -- 172.21.15.181:0/1548498164 >> 172.21.15.181:0/1548498164 conn(0x7fd974104510 msgr2=0x7fd9741051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:25:41.631 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.089+0000 7fd97c7c9700 1 -- 172.21.15.181:0/1548498164 shutdown_connections 2024-05-03T20:25:41.631 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:41.089+0000 7fd97c7c9700 1 -- 172.21.15.181:0/1548498164 wait complete. 2024-05-03T20:25:41.631 INFO:teuthology.orchestra.run.smithi181.stdout:Generating new minimal ceph.conf... 2024-05-03T20:25:42.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:42 smithi181 ceph-mon[29431]: from='client.? 172.21.15.181:0/3626719646' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:25:42.732 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.057+0000 7f02c8813700 1 Processor -- start 2024-05-03T20:25:42.732 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.057+0000 7f02c8813700 1 -- start start 2024-05-03T20:25:42.732 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.057+0000 7f02c8813700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c4108fa0 0x7f02c41093c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:25:42.732 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.057+0000 7f02c8813700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f02c4109990 con 0x7f02c4108fa0 2024-05-03T20:25:42.732 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.058+0000 7f02c259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c4108fa0 0x7f02c41093c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:25:42.733 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.058+0000 7f02c259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c4108fa0 0x7f02c41093c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35260/0 (socket says 172.21.15.181:35260) 2024-05-03T20:25:42.733 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.058+0000 7f02c259c700 1 -- 172.21.15.181:0/264249139 learned_addr learned my addr 172.21.15.181:0/264249139 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:25:42.733 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.058+0000 7f02c259c700 1 -- 172.21.15.181:0/264249139 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c410a1a0 con 0x7f02c4108fa0 2024-05-03T20:25:42.733 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.058+0000 7f02c259c700 1 --2- 172.21.15.181:0/264249139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c4108fa0 0x7f02c41093c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0098a0 tx=0x7f02bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9858abf569d99e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:25:42.733 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.058+0000 7f02c159a700 1 -- 172.21.15.181:0/264249139 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02bc004160 con 0x7f02c4108fa0 2024-05-03T20:25:42.733 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.058+0000 7f02c159a700 1 -- 172.21.15.181:0/264249139 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f02bc004300 con 0x7f02c4108fa0 2024-05-03T20:25:42.733 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.058+0000 7f02c159a700 1 -- 172.21.15.181:0/264249139 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02bc011af0 con 0x7f02c4108fa0 2024-05-03T20:25:42.733 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.059+0000 7f02c8813700 1 -- 172.21.15.181:0/264249139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c4108fa0 msgr2=0x7f02c41093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:25:42.733 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.059+0000 7f02c8813700 1 --2- 172.21.15.181:0/264249139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c4108fa0 0x7f02c41093c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0098a0 tx=0x7f02bc009bb0 comp rx=0 tx=0).stop 2024-05-03T20:25:42.733 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.059+0000 7f02c8813700 1 -- 172.21.15.181:0/264249139 shutdown_connections 2024-05-03T20:25:42.733 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.059+0000 7f02c8813700 1 --2- 172.21.15.181:0/264249139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c4108fa0 0x7f02c41093c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:25:42.733 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.059+0000 7f02c8813700 1 -- 172.21.15.181:0/264249139 >> 172.21.15.181:0/264249139 conn(0x7f02c4104560 msgr2=0x7f02c4106980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:25:42.733 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.059+0000 7f02c8813700 1 -- 172.21.15.181:0/264249139 shutdown_connections 2024-05-03T20:25:42.734 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.059+0000 7f02c8813700 1 -- 172.21.15.181:0/264249139 wait complete. 2024-05-03T20:25:42.734 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.060+0000 7f02c8813700 1 Processor -- start 2024-05-03T20:25:42.734 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.060+0000 7f02c8813700 1 -- start start 2024-05-03T20:25:42.734 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.060+0000 7f02c8813700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c4108fa0 0x7f02c407da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:25:42.734 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.060+0000 7f02c8813700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f02c410d3d0 con 0x7f02c4108fa0 2024-05-03T20:25:42.734 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.060+0000 7f02c259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c4108fa0 0x7f02c407da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:25:42.734 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.060+0000 7f02c259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c4108fa0 0x7f02c407da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35264/0 (socket says 172.21.15.181:35264) 2024-05-03T20:25:42.734 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.060+0000 7f02c259c700 1 -- 172.21.15.181:0/3626719646 learned_addr learned my addr 172.21.15.181:0/3626719646 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:25:42.734 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.060+0000 7f02c259c700 1 -- 172.21.15.181:0/3626719646 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c407df80 con 0x7f02c4108fa0 2024-05-03T20:25:42.734 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.060+0000 7f02c259c700 1 --2- 172.21.15.181:0/3626719646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c4108fa0 0x7f02c407da40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f02bc009fb0 tx=0x7f02bc011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:25:42.734 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.060+0000 7f02b37fe700 1 -- 172.21.15.181:0/3626719646 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02bc0036a0 con 0x7f02c4108fa0 2024-05-03T20:25:42.734 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.060+0000 7f02b37fe700 1 -- 172.21.15.181:0/3626719646 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f02bc01a430 con 0x7f02c4108fa0 2024-05-03T20:25:42.734 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.061+0000 7f02c8813700 1 -- 172.21.15.181:0/3626719646 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c407c170 con 0x7f02c4108fa0 2024-05-03T20:25:42.734 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.061+0000 7f02b37fe700 1 -- 172.21.15.181:0/3626719646 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02bc019550 con 0x7f02c4108fa0 2024-05-03T20:25:42.735 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.061+0000 7f02c8813700 1 -- 172.21.15.181:0/3626719646 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c407c6b0 con 0x7f02c4108fa0 2024-05-03T20:25:42.735 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.061+0000 7f02b37fe700 1 -- 172.21.15.181:0/3626719646 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f02bc019be0 con 0x7f02c4108fa0 2024-05-03T20:25:42.735 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.061+0000 7f02b37fe700 1 -- 172.21.15.181:0/3626719646 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f02bc024730 con 0x7f02c4108fa0 2024-05-03T20:25:42.735 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.062+0000 7f02c8813700 1 -- 172.21.15.181:0/3626719646 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c410d550 con 0x7f02c4108fa0 2024-05-03T20:25:42.735 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.064+0000 7f02b37fe700 1 -- 172.21.15.181:0/3626719646 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f02bc043e50 con 0x7f02c4108fa0 2024-05-03T20:25:42.735 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.130+0000 7f02c8813700 1 -- 172.21.15.181:0/3626719646 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f02c407cda0 con 0x7f02c4108fa0 2024-05-03T20:25:42.735 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.131+0000 7f02b37fe700 1 -- 172.21.15.181:0/3626719646 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f02bc032020 con 0x7f02c4108fa0 2024-05-03T20:25:42.735 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.132+0000 7f02c8813700 1 -- 172.21.15.181:0/3626719646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c4108fa0 msgr2=0x7f02c407da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:25:42.735 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.132+0000 7f02c8813700 1 --2- 172.21.15.181:0/3626719646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c4108fa0 0x7f02c407da40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f02bc009fb0 tx=0x7f02bc011940 comp rx=0 tx=0).stop 2024-05-03T20:25:42.735 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.132+0000 7f02c8813700 1 -- 172.21.15.181:0/3626719646 shutdown_connections 2024-05-03T20:25:42.735 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.132+0000 7f02c8813700 1 --2- 172.21.15.181:0/3626719646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c4108fa0 0x7f02c407da40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:25:42.735 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.132+0000 7f02c8813700 1 -- 172.21.15.181:0/3626719646 >> 172.21.15.181:0/3626719646 conn(0x7f02c4104560 msgr2=0x7f02c4105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:25:42.735 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.132+0000 7f02c8813700 1 -- 172.21.15.181:0/3626719646 shutdown_connections 2024-05-03T20:25:42.735 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:42.132+0000 7f02c8813700 1 -- 172.21.15.181:0/3626719646 wait complete. 2024-05-03T20:25:42.735 INFO:teuthology.orchestra.run.smithi181.stdout:Restarting the monitor... 2024-05-03T20:25:42.982 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:42 smithi181 systemd[1]: Stopping Ceph mon.a for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:25:43.332 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:42 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-a[29427]: 2024-05-03T20:25:42.983+0000 7f4f3e562700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:25:43.333 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:42 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-a[29427]: 2024-05-03T20:25:42.983+0000 7f4f3e562700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-03T20:25:43.333 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:43 smithi181 podman[29676]: 2024-05-03 20:25:43.116534411 +0000 UTC m=+0.217046939 container died a5df4e249393287f2eb5b1e16e4d1910359c5511159f5af05ba822a885a42a15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-a, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T20:25:43.649 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:43 smithi181 podman[29676]: 2024-05-03 20:25:43.433113512 +0000 UTC m=+0.533626040 container cleanup a5df4e249393287f2eb5b1e16e4d1910359c5511159f5af05ba822a885a42a15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-03T20:25:43.649 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:43 smithi181 bash[29676]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-a 2024-05-03T20:25:43.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:43 smithi181 podman[29690]: 2024-05-03 20:25:43.650542648 +0000 UTC m=+0.528849029 container remove a5df4e249393287f2eb5b1e16e4d1910359c5511159f5af05ba822a885a42a15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-a, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-03T20:25:43.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:43 smithi181 systemd[1]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-03T20:25:43.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:43 smithi181 systemd[1]: Stopped Ceph mon.a for 1cd058ee-098b-11ef-bc97-c7b262605968. 2024-05-03T20:25:43.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:43 smithi181 systemd[1]: Starting Ceph mon.a for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:25:44.319 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 podman[29781]: 2024-05-03 20:25:43.966053096 +0000 UTC m=+0.018537889 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T20:25:44.319 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 podman[29781]: 2024-05-03 20:25:44.118313946 +0000 UTC m=+0.170798721 container create e1d7a382742ab709b40f3a0b5a0a07e7841de2ef94fa45bf0c0fb342d174427d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-03T20:25:44.319 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 podman[29781]: 2024-05-03 20:25:44.320352224 +0000 UTC m=+0.372837000 container init e1d7a382742ab709b40f3a0b5a0a07e7841de2ef94fa45bf0c0fb342d174427d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS) 2024-05-03T20:25:44.416 INFO:teuthology.orchestra.run.smithi181.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-03T20:25:44.579 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 podman[29781]: 2024-05-03 20:25:44.323209356 +0000 UTC m=+0.375694132 container start e1d7a382742ab709b40f3a0b5a0a07e7841de2ef94fa45bf0c0fb342d174427d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T20:25:44.579 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: set uid:gid to 167:167 (ceph:ceph) 2024-05-03T20:25:44.579 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev), process ceph-mon, pid 2 2024-05-03T20:25:44.579 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: pidfile_write: ignore empty --pid-file 2024-05-03T20:25:44.579 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: load: jerasure load: lrc 2024-05-03T20:25:44.580 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: RocksDB version: 7.9.2 2024-05-03T20:25:44.580 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Git sha 0 2024-05-03T20:25:44.580 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Compile date 2024-05-01 21:12:03 2024-05-03T20:25:44.580 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: DB SUMMARY 2024-05-03T20:25:44.580 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: DB Session ID: N7XVO36NGFWG0KE4MS3H 2024-05-03T20:25:44.580 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: CURRENT file: CURRENT 2024-05-03T20:25:44.580 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: IDENTITY file: IDENTITY 2024-05-03T20:25:44.580 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-03T20:25:44.580 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-03T20:25:44.580 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105032 ; 2024-05-03T20:25:44.580 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.error_if_exists: 0 2024-05-03T20:25:44.580 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.create_if_missing: 0 2024-05-03T20:25:44.580 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.paranoid_checks: 1 2024-05-03T20:25:44.580 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-03T20:25:44.580 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-03T20:25:44.581 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-03T20:25:44.581 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.env: 0x5610149a0cc0 2024-05-03T20:25:44.581 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.fs: PosixFileSystem 2024-05-03T20:25:44.581 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.info_log: 0x56101523fda0 2024-05-03T20:25:44.581 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.max_file_opening_threads: 16 2024-05-03T20:25:44.581 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.statistics: (nil) 2024-05-03T20:25:44.581 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.use_fsync: 0 2024-05-03T20:25:44.581 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.max_log_file_size: 0 2024-05-03T20:25:44.581 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-03T20:25:44.581 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-03T20:25:44.581 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.keep_log_file_num: 1000 2024-05-03T20:25:44.581 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.recycle_log_file_num: 0 2024-05-03T20:25:44.581 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.allow_fallocate: 1 2024-05-03T20:25:44.583 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.allow_mmap_reads: 0 2024-05-03T20:25:44.583 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.allow_mmap_writes: 0 2024-05-03T20:25:44.583 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.use_direct_reads: 0 2024-05-03T20:25:44.583 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-03T20:25:44.583 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.create_missing_column_families: 0 2024-05-03T20:25:44.583 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.db_log_dir: 2024-05-03T20:25:44.583 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.wal_dir: 2024-05-03T20:25:44.583 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-03T20:25:44.583 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-03T20:25:44.583 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-03T20:25:44.583 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-03T20:25:44.584 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-03T20:25:44.584 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-03T20:25:44.584 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.advise_random_on_open: 1 2024-05-03T20:25:44.584 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.db_write_buffer_size: 0 2024-05-03T20:25:44.584 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.write_buffer_manager: 0x5610152110e0 2024-05-03T20:25:44.584 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-03T20:25:44.584 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-03T20:25:44.584 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-03T20:25:44.584 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.rate_limiter: (nil) 2024-05-03T20:25:44.584 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-03T20:25:44.584 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.wal_recovery_mode: 2 2024-05-03T20:25:44.584 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.enable_thread_tracking: 0 2024-05-03T20:25:44.584 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.enable_pipelined_write: 0 2024-05-03T20:25:44.584 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.unordered_write: 0 2024-05-03T20:25:44.584 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-03T20:25:44.584 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-03T20:25:44.584 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-03T20:25:44.585 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-03T20:25:44.585 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.row_cache: None 2024-05-03T20:25:44.585 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.wal_filter: None 2024-05-03T20:25:44.585 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-03T20:25:44.585 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.allow_ingest_behind: 0 2024-05-03T20:25:44.585 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.two_write_queues: 0 2024-05-03T20:25:44.585 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.manual_wal_flush: 0 2024-05-03T20:25:44.585 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.wal_compression: 0 2024-05-03T20:25:44.585 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.atomic_flush: 0 2024-05-03T20:25:44.585 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-03T20:25:44.585 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-03T20:25:44.585 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-03T20:25:44.585 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.log_readahead_size: 0 2024-05-03T20:25:44.585 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-03T20:25:44.585 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.best_efforts_recovery: 0 2024-05-03T20:25:44.586 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-03T20:25:44.586 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-03T20:25:44.586 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.allow_data_in_errors: 0 2024-05-03T20:25:44.586 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.db_host_id: __hostname__ 2024-05-03T20:25:44.586 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-03T20:25:44.586 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.max_background_jobs: 2 2024-05-03T20:25:44.586 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.max_background_compactions: -1 2024-05-03T20:25:44.586 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.max_subcompactions: 1 2024-05-03T20:25:44.586 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-03T20:25:44.586 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-03T20:25:44.586 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-03T20:25:44.586 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.max_total_wal_size: 0 2024-05-03T20:25:44.586 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-03T20:25:44.586 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-03T20:25:44.586 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-03T20:25:44.587 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-03T20:25:44.587 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.max_open_files: -1 2024-05-03T20:25:44.587 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.bytes_per_sync: 0 2024-05-03T20:25:44.587 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-03T20:25:44.587 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-03T20:25:44.587 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.compaction_readahead_size: 0 2024-05-03T20:25:44.587 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.max_background_flushes: -1 2024-05-03T20:25:44.587 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Compression algorithms supported: 2024-05-03T20:25:44.587 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-03T20:25:44.587 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: kZSTD supported: 0 2024-05-03T20:25:44.587 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: kXpressCompression supported: 0 2024-05-03T20:25:44.587 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: kLZ4HCCompression supported: 1 2024-05-03T20:25:44.587 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: kZlibCompression supported: 1 2024-05-03T20:25:44.587 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: kSnappyCompression supported: 1 2024-05-03T20:25:44.587 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: kLZ4Compression supported: 1 2024-05-03T20:25:44.588 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: kBZip2Compression supported: 0 2024-05-03T20:25:44.588 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-03T20:25:44.588 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-03T20:25:44.588 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-03T20:25:44.588 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-03T20:25:44.588 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-03T20:25:44.588 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.merge_operator: 2024-05-03T20:25:44.588 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.compaction_filter: None 2024-05-03T20:25:44.588 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.compaction_filter_factory: None 2024-05-03T20:25:44.588 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.sst_partitioner_factory: None 2024-05-03T20:25:44.588 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-03T20:25:44.588 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-03T20:25:44.588 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56101523ff60) 2024-05-03T20:25:44.588 INFO:journalctl@ceph.mon.a.smithi181.stdout: cache_index_and_filter_blocks: 1 2024-05-03T20:25:44.588 INFO:journalctl@ceph.mon.a.smithi181.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-03T20:25:44.589 INFO:journalctl@ceph.mon.a.smithi181.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-03T20:25:44.589 INFO:journalctl@ceph.mon.a.smithi181.stdout: pin_top_level_index_and_filter: 1 2024-05-03T20:25:44.589 INFO:journalctl@ceph.mon.a.smithi181.stdout: index_type: 0 2024-05-03T20:25:44.589 INFO:journalctl@ceph.mon.a.smithi181.stdout: data_block_index_type: 0 2024-05-03T20:25:44.589 INFO:journalctl@ceph.mon.a.smithi181.stdout: index_shortening: 1 2024-05-03T20:25:44.589 INFO:journalctl@ceph.mon.a.smithi181.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-03T20:25:44.589 INFO:journalctl@ceph.mon.a.smithi181.stdout: checksum: 4 2024-05-03T20:25:44.589 INFO:journalctl@ceph.mon.a.smithi181.stdout: no_block_cache: 0 2024-05-03T20:25:44.589 INFO:journalctl@ceph.mon.a.smithi181.stdout: block_cache: 0x5610152431f0 2024-05-03T20:25:44.589 INFO:journalctl@ceph.mon.a.smithi181.stdout: block_cache_name: BinnedLRUCache 2024-05-03T20:25:44.589 INFO:journalctl@ceph.mon.a.smithi181.stdout: block_cache_options: 2024-05-03T20:25:44.589 INFO:journalctl@ceph.mon.a.smithi181.stdout: capacity : 536870912 2024-05-03T20:25:44.589 INFO:journalctl@ceph.mon.a.smithi181.stdout: num_shard_bits : 4 2024-05-03T20:25:44.589 INFO:journalctl@ceph.mon.a.smithi181.stdout: strict_capacity_limit : 0 2024-05-03T20:25:44.589 INFO:journalctl@ceph.mon.a.smithi181.stdout: high_pri_pool_ratio: 0.000 2024-05-03T20:25:44.589 INFO:journalctl@ceph.mon.a.smithi181.stdout: block_cache_compressed: (nil) 2024-05-03T20:25:44.590 INFO:journalctl@ceph.mon.a.smithi181.stdout: persistent_cache: (nil) 2024-05-03T20:25:44.590 INFO:journalctl@ceph.mon.a.smithi181.stdout: block_size: 4096 2024-05-03T20:25:44.590 INFO:journalctl@ceph.mon.a.smithi181.stdout: block_size_deviation: 10 2024-05-03T20:25:44.590 INFO:journalctl@ceph.mon.a.smithi181.stdout: block_restart_interval: 16 2024-05-03T20:25:44.590 INFO:journalctl@ceph.mon.a.smithi181.stdout: index_block_restart_interval: 1 2024-05-03T20:25:44.590 INFO:journalctl@ceph.mon.a.smithi181.stdout: metadata_block_size: 4096 2024-05-03T20:25:44.590 INFO:journalctl@ceph.mon.a.smithi181.stdout: partition_filters: 0 2024-05-03T20:25:44.590 INFO:journalctl@ceph.mon.a.smithi181.stdout: use_delta_encoding: 1 2024-05-03T20:25:44.590 INFO:journalctl@ceph.mon.a.smithi181.stdout: filter_policy: bloomfilter 2024-05-03T20:25:44.590 INFO:journalctl@ceph.mon.a.smithi181.stdout: whole_key_filtering: 1 2024-05-03T20:25:44.590 INFO:journalctl@ceph.mon.a.smithi181.stdout: verify_compression: 0 2024-05-03T20:25:44.590 INFO:journalctl@ceph.mon.a.smithi181.stdout: read_amp_bytes_per_bit: 0 2024-05-03T20:25:44.590 INFO:journalctl@ceph.mon.a.smithi181.stdout: format_version: 5 2024-05-03T20:25:44.591 INFO:journalctl@ceph.mon.a.smithi181.stdout: enable_index_compression: 1 2024-05-03T20:25:44.591 INFO:journalctl@ceph.mon.a.smithi181.stdout: block_align: 0 2024-05-03T20:25:44.591 INFO:journalctl@ceph.mon.a.smithi181.stdout: max_auto_readahead_size: 262144 2024-05-03T20:25:44.591 INFO:journalctl@ceph.mon.a.smithi181.stdout: prepopulate_block_cache: 0 2024-05-03T20:25:44.591 INFO:journalctl@ceph.mon.a.smithi181.stdout: initial_auto_readahead_size: 8192 2024-05-03T20:25:44.591 INFO:journalctl@ceph.mon.a.smithi181.stdout: num_file_reads_for_auto_readahead: 2 2024-05-03T20:25:44.591 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.write_buffer_size: 33554432 2024-05-03T20:25:44.591 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.max_write_buffer_number: 2 2024-05-03T20:25:44.591 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.compression: NoCompression 2024-05-03T20:25:44.591 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.bottommost_compression: Disabled 2024-05-03T20:25:44.591 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.prefix_extractor: nullptr 2024-05-03T20:25:44.591 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-03T20:25:44.591 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.num_levels: 7 2024-05-03T20:25:44.591 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-03T20:25:44.591 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-03T20:25:44.592 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-03T20:25:44.592 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-03T20:25:44.592 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-03T20:25:44.592 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-03T20:25:44.592 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-03T20:25:44.592 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-03T20:25:44.592 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-03T20:25:44.592 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-03T20:25:44.592 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-03T20:25:44.592 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-03T20:25:44.592 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-03T20:25:44.592 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.compression_opts.level: 32767 2024-05-03T20:25:44.592 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.compression_opts.strategy: 0 2024-05-03T20:25:44.592 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-03T20:25:44.592 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-03T20:25:44.593 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-03T20:25:44.593 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-03T20:25:44.593 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.compression_opts.enabled: false 2024-05-03T20:25:44.593 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-03T20:25:44.593 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-03T20:25:44.593 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-03T20:25:44.593 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-03T20:25:44.593 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.target_file_size_base: 67108864 2024-05-03T20:25:44.593 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-03T20:25:44.593 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-03T20:25:44.593 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-03T20:25:44.593 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-03T20:25:44.593 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-03T20:25:44.593 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-03T20:25:44.593 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-03T20:25:44.593 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-03T20:25:44.594 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-03T20:25:44.594 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-03T20:25:44.594 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-03T20:25:44.594 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-03T20:25:44.594 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-03T20:25:44.594 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-03T20:25:44.594 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.arena_block_size: 1048576 2024-05-03T20:25:44.594 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-03T20:25:44.594 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-03T20:25:44.594 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.disable_auto_compactions: 0 2024-05-03T20:25:44.594 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-03T20:25:44.594 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-03T20:25:44.594 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-03T20:25:44.594 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-03T20:25:44.595 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-03T20:25:44.595 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-03T20:25:44.595 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-03T20:25:44.595 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-03T20:25:44.595 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-03T20:25:44.595 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-03T20:25:44.595 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-03T20:25:44.595 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.inplace_update_support: 0 2024-05-03T20:25:44.595 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-03T20:25:44.595 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-03T20:25:44.595 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-03T20:25:44.595 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-03T20:25:44.595 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.bloom_locality: 0 2024-05-03T20:25:44.595 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.max_successive_merges: 0 2024-05-03T20:25:44.595 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-03T20:25:44.596 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.paranoid_file_checks: 0 2024-05-03T20:25:44.596 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.force_consistency_checks: 1 2024-05-03T20:25:44.596 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.report_bg_io_stats: 0 2024-05-03T20:25:44.596 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.ttl: 2592000 2024-05-03T20:25:44.596 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-03T20:25:44.596 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-03T20:25:44.596 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-03T20:25:44.596 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.enable_blob_files: false 2024-05-03T20:25:44.596 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.min_blob_size: 0 2024-05-03T20:25:44.596 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.blob_file_size: 268435456 2024-05-03T20:25:44.596 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-03T20:25:44.596 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-03T20:25:44.596 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-03T20:25:44.596 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-03T20:25:44.597 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-03T20:25:44.597 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.blob_file_starting_level: 0 2024-05-03T20:25:44.597 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-03T20:25:44.597 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-03T20:25:44.597 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-03T20:25:44.597 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f5f2a4fa-36d3-4664-8e44-3008d118eaf9 2024-05-03T20:25:44.597 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714767944352084, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-03T20:25:44.597 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-03T20:25:44.597 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714767944353044, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100209, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 98093, "index_size": 308, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14161, "raw_average_key_size": 49, "raw_value_size": 90376, "raw_average_value_size": 317, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714767944, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f5f2a4fa-36d3-4664-8e44-3008d118eaf9", "db_session_id": "N7XVO36NGFWG0KE4MS3H", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-03T20:25:44.597 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714767944353127, "job": 1, "event": "recovery_finished"} 2024-05-03T20:25:44.597 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-03T20:25:44.597 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-03T20:25:44.597 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561015360000 2024-05-03T20:25:44.597 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: DB pointer 0x56101534c000 2024-05-03T20:25:44.597 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-03T20:25:44.598 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-03T20:25:44.598 INFO:journalctl@ceph.mon.a.smithi181.stdout: ** DB Stats ** 2024-05-03T20:25:44.598 INFO:journalctl@ceph.mon.a.smithi181.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-03T20:25:44.598 INFO:journalctl@ceph.mon.a.smithi181.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-03T20:25:44.598 INFO:journalctl@ceph.mon.a.smithi181.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-03T20:25:44.598 INFO:journalctl@ceph.mon.a.smithi181.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-03T20:25:44.598 INFO:journalctl@ceph.mon.a.smithi181.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-03T20:25:44.598 INFO:journalctl@ceph.mon.a.smithi181.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-03T20:25:44.598 INFO:journalctl@ceph.mon.a.smithi181.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-03T20:25:44.598 INFO:journalctl@ceph.mon.a.smithi181.stdout: 2024-05-03T20:25:44.598 INFO:journalctl@ceph.mon.a.smithi181.stdout: ** Compaction Stats [default] ** 2024-05-03T20:25:44.598 INFO:journalctl@ceph.mon.a.smithi181.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-03T20:25:44.598 INFO:journalctl@ceph.mon.a.smithi181.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-03T20:25:44.598 INFO:journalctl@ceph.mon.a.smithi181.stdout: L0 2/0 99.72 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 163.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T20:25:44.598 INFO:journalctl@ceph.mon.a.smithi181.stdout: Sum 2/0 99.72 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 163.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T20:25:44.599 INFO:journalctl@ceph.mon.a.smithi181.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 163.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T20:25:44.599 INFO:journalctl@ceph.mon.a.smithi181.stdout: 2024-05-03T20:25:44.599 INFO:journalctl@ceph.mon.a.smithi181.stdout: ** Compaction Stats [default] ** 2024-05-03T20:25:44.599 INFO:journalctl@ceph.mon.a.smithi181.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-03T20:25:44.599 INFO:journalctl@ceph.mon.a.smithi181.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-03T20:25:44.599 INFO:journalctl@ceph.mon.a.smithi181.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 163.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T20:25:44.599 INFO:journalctl@ceph.mon.a.smithi181.stdout: 2024-05-03T20:25:44.599 INFO:journalctl@ceph.mon.a.smithi181.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-03T20:25:44.599 INFO:journalctl@ceph.mon.a.smithi181.stdout: 2024-05-03T20:25:44.599 INFO:journalctl@ceph.mon.a.smithi181.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-03T20:25:44.599 INFO:journalctl@ceph.mon.a.smithi181.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-03T20:25:44.599 INFO:journalctl@ceph.mon.a.smithi181.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-03T20:25:44.599 INFO:journalctl@ceph.mon.a.smithi181.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-03T20:25:44.599 INFO:journalctl@ceph.mon.a.smithi181.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-03T20:25:44.599 INFO:journalctl@ceph.mon.a.smithi181.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-03T20:25:44.599 INFO:journalctl@ceph.mon.a.smithi181.stdout: Cumulative compaction: 0.00 GB write, 32.74 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-03T20:25:44.600 INFO:journalctl@ceph.mon.a.smithi181.stdout: Interval compaction: 0.00 GB write, 32.74 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-03T20:25:44.600 INFO:journalctl@ceph.mon.a.smithi181.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-03T20:25:44.600 INFO:journalctl@ceph.mon.a.smithi181.stdout: Block cache BinnedLRUCache@0x5610152431f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-05-03T20:25:44.600 INFO:journalctl@ceph.mon.a.smithi181.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-03T20:25:44.600 INFO:journalctl@ceph.mon.a.smithi181.stdout: 2024-05-03T20:25:44.600 INFO:journalctl@ceph.mon.a.smithi181.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-03T20:25:44.600 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: starting mon.a rank 0 at public addrs [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] at bind addrs [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:25:44.600 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: mon.a@-1(???) e1 preinit fsid 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:25:44.600 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: mon.a@-1(???).mds e1 new map 2024-05-03T20:25:44.600 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: mon.a@-1(???).mds e1 print_map 2024-05-03T20:25:44.600 INFO:journalctl@ceph.mon.a.smithi181.stdout: e1 2024-05-03T20:25:44.600 INFO:journalctl@ceph.mon.a.smithi181.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-03T20:25:44.600 INFO:journalctl@ceph.mon.a.smithi181.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-03T20:25:44.600 INFO:journalctl@ceph.mon.a.smithi181.stdout: legacy client fscid: -1 2024-05-03T20:25:44.600 INFO:journalctl@ceph.mon.a.smithi181.stdout: 2024-05-03T20:25:44.601 INFO:journalctl@ceph.mon.a.smithi181.stdout: No filesystems configured 2024-05-03T20:25:44.601 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-03T20:25:44.601 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T20:25:44.601 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T20:25:44.601 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T20:25:44.601 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-03T20:25:44.601 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 bash[29781]: e1d7a382742ab709b40f3a0b5a0a07e7841de2ef94fa45bf0c0fb342d174427d 2024-05-03T20:25:44.601 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-03T20:25:44.601 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: monmap epoch 1 2024-05-03T20:25:44.601 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: fsid 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:25:44.601 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: last_changed 2024-05-03T20:25:36.511893+0000 2024-05-03T20:25:44.601 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: created 2024-05-03T20:25:36.511893+0000 2024-05-03T20:25:44.601 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: min_mon_release 19 (squid) 2024-05-03T20:25:44.601 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: election_strategy: 1 2024-05-03T20:25:44.601 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: 0: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.a 2024-05-03T20:25:44.601 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: fsmap 2024-05-03T20:25:44.601 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: osdmap e1: 0 total, 0 up, 0 in 2024-05-03T20:25:44.601 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 ceph-mon[29801]: mgrmap e1: no daemons active 2024-05-03T20:25:44.602 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:44 smithi181 systemd[1]: Started Ceph mon.a for 1cd058ee-098b-11ef-bc97-c7b262605968. 2024-05-03T20:25:45.492 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.925+0000 7fbc8a075700 1 Processor -- start 2024-05-03T20:25:45.492 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.925+0000 7fbc8a075700 1 -- start start 2024-05-03T20:25:45.492 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.925+0000 7fbc8a075700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc84107e50 0x7fbc84108270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:25:45.492 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.925+0000 7fbc8a075700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbc841087b0 con 0x7fbc84107e50 2024-05-03T20:25:45.492 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.925+0000 7fbc837fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc84107e50 0x7fbc84108270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:25:45.492 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.925+0000 7fbc837fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc84107e50 0x7fbc84108270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35276/0 (socket says 172.21.15.181:35276) 2024-05-03T20:25:45.492 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.925+0000 7fbc837fe700 1 -- 172.21.15.181:0/2362649515 learned_addr learned my addr 172.21.15.181:0/2362649515 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:25:45.492 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.926+0000 7fbc837fe700 1 -- 172.21.15.181:0/2362649515 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc84109040 con 0x7fbc84107e50 2024-05-03T20:25:45.492 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.926+0000 7fbc837fe700 1 --2- 172.21.15.181:0/2362649515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc84107e50 0x7fbc84108270 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbc740098a0 tx=0x7fbc74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ae60cdd3f3288e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:25:45.492 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.926+0000 7fbc827fc700 1 -- 172.21.15.181:0/2362649515 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc74004160 con 0x7fbc84107e50 2024-05-03T20:25:45.492 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.926+0000 7fbc827fc700 1 -- 172.21.15.181:0/2362649515 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbc74004300 con 0x7fbc84107e50 2024-05-03T20:25:45.493 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.926+0000 7fbc827fc700 1 -- 172.21.15.181:0/2362649515 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc74011af0 con 0x7fbc84107e50 2024-05-03T20:25:45.493 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.926+0000 7fbc8a075700 1 -- 172.21.15.181:0/2362649515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc84107e50 msgr2=0x7fbc84108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:25:45.493 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.926+0000 7fbc8a075700 1 --2- 172.21.15.181:0/2362649515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc84107e50 0x7fbc84108270 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbc740098a0 tx=0x7fbc74009bb0 comp rx=0 tx=0).stop 2024-05-03T20:25:45.493 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.927+0000 7fbc8a075700 1 -- 172.21.15.181:0/2362649515 shutdown_connections 2024-05-03T20:25:45.493 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.927+0000 7fbc8a075700 1 --2- 172.21.15.181:0/2362649515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc84107e50 0x7fbc84108270 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:25:45.493 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.927+0000 7fbc8a075700 1 -- 172.21.15.181:0/2362649515 >> 172.21.15.181:0/2362649515 conn(0x7fbc84103460 msgr2=0x7fbc84105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:25:45.493 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.927+0000 7fbc8a075700 1 -- 172.21.15.181:0/2362649515 shutdown_connections 2024-05-03T20:25:45.493 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.927+0000 7fbc8a075700 1 -- 172.21.15.181:0/2362649515 wait complete. 2024-05-03T20:25:45.493 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.927+0000 7fbc8a075700 1 Processor -- start 2024-05-03T20:25:45.493 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.927+0000 7fbc8a075700 1 -- start start 2024-05-03T20:25:45.493 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.927+0000 7fbc8a075700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc84107e50 0x7fbc8419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:25:45.493 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.927+0000 7fbc8a075700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbc84108930 con 0x7fbc84107e50 2024-05-03T20:25:45.493 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.928+0000 7fbc837fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc84107e50 0x7fbc8419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:25:45.493 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.928+0000 7fbc837fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc84107e50 0x7fbc8419ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35278/0 (socket says 172.21.15.181:35278) 2024-05-03T20:25:45.493 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.928+0000 7fbc837fe700 1 -- 172.21.15.181:0/3659001714 learned_addr learned my addr 172.21.15.181:0/3659001714 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:25:45.493 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.928+0000 7fbc837fe700 1 -- 172.21.15.181:0/3659001714 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8419d370 con 0x7fbc84107e50 2024-05-03T20:25:45.494 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.928+0000 7fbc837fe700 1 --2- 172.21.15.181:0/3659001714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc84107e50 0x7fbc8419ce30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbc74003f40 tx=0x7fbc74011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:25:45.494 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.928+0000 7fbc80ff9700 1 -- 172.21.15.181:0/3659001714 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc740038b0 con 0x7fbc84107e50 2024-05-03T20:25:45.494 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.928+0000 7fbc80ff9700 1 -- 172.21.15.181:0/3659001714 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbc74024430 con 0x7fbc84107e50 2024-05-03T20:25:45.494 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.928+0000 7fbc8a075700 1 -- 172.21.15.181:0/3659001714 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc8419d600 con 0x7fbc84107e50 2024-05-03T20:25:45.494 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.928+0000 7fbc80ff9700 1 -- 172.21.15.181:0/3659001714 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc7401a430 con 0x7fbc84107e50 2024-05-03T20:25:45.494 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.928+0000 7fbc8a075700 1 -- 172.21.15.181:0/3659001714 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc841a02f0 con 0x7fbc84107e50 2024-05-03T20:25:45.494 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.928+0000 7fbc80ff9700 1 -- 172.21.15.181:0/3659001714 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbc7401aac0 con 0x7fbc84107e50 2024-05-03T20:25:45.494 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.929+0000 7fbc80ff9700 1 -- 172.21.15.181:0/3659001714 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbc7400b990 con 0x7fbc84107e50 2024-05-03T20:25:45.494 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.929+0000 7fbc8a075700 1 -- 172.21.15.181:0/3659001714 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc84064b00 con 0x7fbc84107e50 2024-05-03T20:25:45.494 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:44.932+0000 7fbc80ff9700 1 -- 172.21.15.181:0/3659001714 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fbc74024990 con 0x7fbc84107e50 2024-05-03T20:25:45.494 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:45.001+0000 7fbc8a075700 1 -- 172.21.15.181:0/3659001714 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fbc8402e850 con 0x7fbc84107e50 2024-05-03T20:25:45.494 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:45.003+0000 7fbc80ff9700 1 -- 172.21.15.181:0/3659001714 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fbc740245e0 con 0x7fbc84107e50 2024-05-03T20:25:45.494 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:45.003+0000 7fbc8a075700 1 -- 172.21.15.181:0/3659001714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc84107e50 msgr2=0x7fbc8419ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:25:45.494 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:45.003+0000 7fbc8a075700 1 --2- 172.21.15.181:0/3659001714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc84107e50 0x7fbc8419ce30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbc74003f40 tx=0x7fbc74011890 comp rx=0 tx=0).stop 2024-05-03T20:25:45.494 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:45.004+0000 7fbc8a075700 1 -- 172.21.15.181:0/3659001714 shutdown_connections 2024-05-03T20:25:45.495 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:45.004+0000 7fbc8a075700 1 --2- 172.21.15.181:0/3659001714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc84107e50 0x7fbc8419ce30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:25:45.495 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:45.004+0000 7fbc8a075700 1 -- 172.21.15.181:0/3659001714 >> 172.21.15.181:0/3659001714 conn(0x7fbc84103460 msgr2=0x7fbc84104290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:25:45.495 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:45.004+0000 7fbc8a075700 1 -- 172.21.15.181:0/3659001714 shutdown_connections 2024-05-03T20:25:45.495 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:45.004+0000 7fbc8a075700 1 -- 172.21.15.181:0/3659001714 wait complete. 2024-05-03T20:25:45.495 INFO:teuthology.orchestra.run.smithi181.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-03T20:25:45.517 INFO:teuthology.orchestra.run.smithi181.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-03T20:25:45.518 INFO:teuthology.orchestra.run.smithi181.stdout:Creating mgr... 2024-05-03T20:25:45.518 INFO:teuthology.orchestra.run.smithi181.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-03T20:25:45.518 INFO:teuthology.orchestra.run.smithi181.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-03T20:25:45.761 INFO:teuthology.orchestra.run.smithi181.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mgr.x 2024-05-03T20:25:45.761 INFO:teuthology.orchestra.run.smithi181.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mgr.x.service: Unit ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mgr.x.service not loaded. 2024-05-03T20:25:45.911 INFO:teuthology.orchestra.run.smithi181.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1cd058ee-098b-11ef-bc97-c7b262605968.target.wants/ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-1cd058ee-098b-11ef-bc97-c7b262605968@.service. 2024-05-03T20:25:46.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:46 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3659001714' entity='client.admin' 2024-05-03T20:25:46.214 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:46 smithi181 podman[30018]: 2024-05-03 20:25:46.154588672 +0000 UTC m=+0.121130439 container create 87c0517d69b19f49fd5b892b63c53789fc6a1a8eb5402683d1f28f956fc5805f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-05-03T20:25:46.452 INFO:teuthology.orchestra.run.smithi181.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-03T20:25:46.452 INFO:teuthology.orchestra.run.smithi181.stdout:systemctl: stdout disabled 2024-05-03T20:25:46.459 INFO:teuthology.orchestra.run.smithi181.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-03T20:25:46.459 INFO:teuthology.orchestra.run.smithi181.stdout:systemctl: stdout inactive 2024-05-03T20:25:46.459 INFO:teuthology.orchestra.run.smithi181.stdout:firewalld.service is not enabled 2024-05-03T20:25:46.459 INFO:teuthology.orchestra.run.smithi181.stdout:Not possible to enable service . firewalld.service is not available 2024-05-03T20:25:46.467 INFO:teuthology.orchestra.run.smithi181.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-03T20:25:46.467 INFO:teuthology.orchestra.run.smithi181.stdout:systemctl: stdout disabled 2024-05-03T20:25:46.475 INFO:teuthology.orchestra.run.smithi181.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-03T20:25:46.475 INFO:teuthology.orchestra.run.smithi181.stdout:systemctl: stdout inactive 2024-05-03T20:25:46.475 INFO:teuthology.orchestra.run.smithi181.stdout:firewalld.service is not enabled 2024-05-03T20:25:46.475 INFO:teuthology.orchestra.run.smithi181.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-03T20:25:46.475 INFO:teuthology.orchestra.run.smithi181.stdout:Waiting for mgr to start... 2024-05-03T20:25:46.475 INFO:teuthology.orchestra.run.smithi181.stdout:Waiting for mgr... 2024-05-03T20:25:46.524 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:46 smithi181 podman[30018]: 2024-05-03 20:25:46.356648463 +0000 UTC m=+0.323190188 container init 87c0517d69b19f49fd5b892b63c53789fc6a1a8eb5402683d1f28f956fc5805f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-03T20:25:46.524 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:46 smithi181 podman[30018]: 2024-05-03 20:25:46.359219567 +0000 UTC m=+0.325761297 container start 87c0517d69b19f49fd5b892b63c53789fc6a1a8eb5402683d1f28f956fc5805f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T20:25:46.524 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:46 smithi181 bash[30018]: 87c0517d69b19f49fd5b892b63c53789fc6a1a8eb5402683d1f28f956fc5805f 2024-05-03T20:25:46.524 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:46 smithi181 systemd[1]: Started Ceph mgr.x for 1cd058ee-098b-11ef-bc97-c7b262605968. 2024-05-03T20:25:46.819 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:46 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:46.524+0000 7f4c9d6f2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T20:25:46.819 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:46 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:46.713+0000 7f4c9d6f2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T20:25:46.819 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:46 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:46.819+0000 7f4c9d6f2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T20:25:47.146 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:47 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/419638829' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-03T20:25:47.463 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:47 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:47.147+0000 7f4c9d6f2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T20:25:47.463 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:47 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:47.232+0000 7f4c9d6f2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T20:25:47.776 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout 2024-05-03T20:25:47.776 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout { 2024-05-03T20:25:47.776 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "fsid": "1cd058ee-098b-11ef-bc97-c7b262605968", 2024-05-03T20:25:47.776 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "health": { 2024-05-03T20:25:47.776 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-03T20:25:47.776 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-03T20:25:47.776 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-03T20:25:47.777 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:47.777 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-03T20:25:47.777 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-03T20:25:47.777 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout 0 2024-05-03T20:25:47.777 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout ], 2024-05-03T20:25:47.777 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-03T20:25:47.777 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "a" 2024-05-03T20:25:47.777 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout ], 2024-05-03T20:25:47.777 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-03T20:25:47.777 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-03T20:25:47.777 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T20:25:47.777 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-03T20:25:47.777 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-03T20:25:47.777 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:47.777 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-03T20:25:47.778 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T20:25:47.778 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-03T20:25:47.778 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-03T20:25:47.778 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-03T20:25:47.786 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-03T20:25:47.786 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-03T20:25:47.786 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-03T20:25:47.786 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:47.786 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-03T20:25:47.787 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-03T20:25:47.787 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-03T20:25:47.787 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-03T20:25:47.787 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-03T20:25:47.787 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-03T20:25:47.787 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-03T20:25:47.787 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-03T20:25:47.787 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-03T20:25:47.787 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:47.787 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-03T20:25:47.787 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T20:25:47.787 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-03T20:25:47.787 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-03T20:25:47.787 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:47.787 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-03T20:25:47.788 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-03T20:25:47.788 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-03T20:25:47.788 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-03T20:25:47.788 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-03T20:25:47.788 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-03T20:25:47.788 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "restful" 2024-05-03T20:25:47.788 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout ], 2024-05-03T20:25:47.788 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T20:25:47.788 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:47.788 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-03T20:25:47.788 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T20:25:47.788 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "modified": "2024-05-03T20:25:39.428264+0000", 2024-05-03T20:25:47.788 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T20:25:47.788 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:47.788 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-03T20:25:47.788 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout } 2024-05-03T20:25:47.789 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.936+0000 7f8a495dc700 1 Processor -- start 2024-05-03T20:25:47.789 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.936+0000 7f8a495dc700 1 -- start start 2024-05-03T20:25:47.789 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.936+0000 7f8a495dc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a44107e50 0x7f8a44108270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:25:47.789 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.936+0000 7f8a495dc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8a441087b0 con 0x7f8a44107e50 2024-05-03T20:25:47.789 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.936+0000 7f8a42ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a44107e50 0x7f8a44108270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:25:47.789 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.936+0000 7f8a42ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a44107e50 0x7f8a44108270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54398/0 (socket says 172.21.15.181:54398) 2024-05-03T20:25:47.789 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.936+0000 7f8a42ffd700 1 -- 172.21.15.181:0/3552794033 learned_addr learned my addr 172.21.15.181:0/3552794033 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:25:47.789 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.937+0000 7f8a42ffd700 1 -- 172.21.15.181:0/3552794033 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a44109040 con 0x7f8a44107e50 2024-05-03T20:25:47.789 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.937+0000 7f8a42ffd700 1 --2- 172.21.15.181:0/3552794033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a44107e50 0x7f8a44108270 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8a340098a0 tx=0x7f8a34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125c24907558d7c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:25:47.789 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.937+0000 7f8a41ffb700 1 -- 172.21.15.181:0/3552794033 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a34004160 con 0x7f8a44107e50 2024-05-03T20:25:47.789 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.937+0000 7f8a41ffb700 1 -- 172.21.15.181:0/3552794033 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8a34004300 con 0x7f8a44107e50 2024-05-03T20:25:47.789 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.937+0000 7f8a41ffb700 1 -- 172.21.15.181:0/3552794033 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a34011af0 con 0x7f8a44107e50 2024-05-03T20:25:47.789 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.937+0000 7f8a495dc700 1 -- 172.21.15.181:0/3552794033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a44107e50 msgr2=0x7f8a44108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:25:47.789 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.937+0000 7f8a495dc700 1 --2- 172.21.15.181:0/3552794033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a44107e50 0x7f8a44108270 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8a340098a0 tx=0x7f8a34009bb0 comp rx=0 tx=0).stop 2024-05-03T20:25:47.789 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.938+0000 7f8a495dc700 1 -- 172.21.15.181:0/3552794033 shutdown_connections 2024-05-03T20:25:47.790 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.938+0000 7f8a495dc700 1 --2- 172.21.15.181:0/3552794033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a44107e50 0x7f8a44108270 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:25:47.790 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.938+0000 7f8a495dc700 1 -- 172.21.15.181:0/3552794033 >> 172.21.15.181:0/3552794033 conn(0x7f8a44103460 msgr2=0x7f8a44105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:25:47.790 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.938+0000 7f8a495dc700 1 -- 172.21.15.181:0/3552794033 shutdown_connections 2024-05-03T20:25:47.790 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.938+0000 7f8a495dc700 1 -- 172.21.15.181:0/3552794033 wait complete. 2024-05-03T20:25:47.790 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.938+0000 7f8a495dc700 1 Processor -- start 2024-05-03T20:25:47.790 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.938+0000 7f8a495dc700 1 -- start start 2024-05-03T20:25:47.790 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.938+0000 7f8a495dc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a44107e50 0x7f8a4407ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:25:47.790 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.938+0000 7f8a495dc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8a44108930 con 0x7f8a44107e50 2024-05-03T20:25:47.790 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.939+0000 7f8a42ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a44107e50 0x7f8a4407ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:25:47.790 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.939+0000 7f8a42ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a44107e50 0x7f8a4407ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54414/0 (socket says 172.21.15.181:54414) 2024-05-03T20:25:47.790 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.939+0000 7f8a42ffd700 1 -- 172.21.15.181:0/419638829 learned_addr learned my addr 172.21.15.181:0/419638829 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:25:47.790 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.939+0000 7f8a42ffd700 1 -- 172.21.15.181:0/419638829 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a4419fb80 con 0x7f8a44107e50 2024-05-03T20:25:47.790 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.939+0000 7f8a42ffd700 1 --2- 172.21.15.181:0/419638829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a44107e50 0x7f8a4407ca30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8a34000c00 tx=0x7f8a34011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:25:47.790 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.939+0000 7f8a2bfff700 1 -- 172.21.15.181:0/419638829 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a34003b20 con 0x7f8a44107e50 2024-05-03T20:25:47.791 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.939+0000 7f8a2bfff700 1 -- 172.21.15.181:0/419638829 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8a34003cc0 con 0x7f8a44107e50 2024-05-03T20:25:47.791 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.939+0000 7f8a495dc700 1 -- 172.21.15.181:0/419638829 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a4419fd50 con 0x7f8a44107e50 2024-05-03T20:25:47.791 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.939+0000 7f8a2bfff700 1 -- 172.21.15.181:0/419638829 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a34024800 con 0x7f8a44107e50 2024-05-03T20:25:47.791 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.939+0000 7f8a495dc700 1 -- 172.21.15.181:0/419638829 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a441a00f0 con 0x7f8a44107e50 2024-05-03T20:25:47.791 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.939+0000 7f8a2bfff700 1 -- 172.21.15.181:0/419638829 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8a34022020 con 0x7f8a44107e50 2024-05-03T20:25:47.791 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.940+0000 7f8a2bfff700 1 -- 172.21.15.181:0/419638829 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8a3402dcf0 con 0x7f8a44107e50 2024-05-03T20:25:47.791 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.940+0000 7f8a495dc700 1 -- 172.21.15.181:0/419638829 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a44064b00 con 0x7f8a44107e50 2024-05-03T20:25:47.791 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:46.943+0000 7f8a2bfff700 1 -- 172.21.15.181:0/419638829 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f8a340042c0 con 0x7f8a44107e50 2024-05-03T20:25:47.791 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:47.017+0000 7f8a495dc700 1 -- 172.21.15.181:0/419638829 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f8a441a0390 con 0x7f8a44107e50 2024-05-03T20:25:47.791 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:47.017+0000 7f8a2bfff700 1 -- 172.21.15.181:0/419638829 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f8a34032030 con 0x7f8a44107e50 2024-05-03T20:25:47.791 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:47.018+0000 7f8a495dc700 1 -- 172.21.15.181:0/419638829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a44107e50 msgr2=0x7f8a4407ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:25:47.791 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:47.018+0000 7f8a495dc700 1 --2- 172.21.15.181:0/419638829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a44107e50 0x7f8a4407ca30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8a34000c00 tx=0x7f8a34011b70 comp rx=0 tx=0).stop 2024-05-03T20:25:47.791 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:47.019+0000 7f8a495dc700 1 -- 172.21.15.181:0/419638829 shutdown_connections 2024-05-03T20:25:47.791 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:47.019+0000 7f8a495dc700 1 --2- 172.21.15.181:0/419638829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a44107e50 0x7f8a4407ca30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:25:47.792 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:47.019+0000 7f8a495dc700 1 -- 172.21.15.181:0/419638829 >> 172.21.15.181:0/419638829 conn(0x7f8a44103460 msgr2=0x7f8a44103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:25:47.792 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:47.019+0000 7f8a495dc700 1 -- 172.21.15.181:0/419638829 shutdown_connections 2024-05-03T20:25:47.792 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:47.019+0000 7f8a495dc700 1 -- 172.21.15.181:0/419638829 wait complete. 2024-05-03T20:25:47.792 INFO:teuthology.orchestra.run.smithi181.stdout:mgr not available, waiting (1/15)... 2024-05-03T20:25:48.713 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:48 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:48.419+0000 7f4c9d6f2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T20:25:48.714 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:48 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:48.585+0000 7f4c9d6f2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T20:25:49.034 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:48 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:48.755+0000 7f4c9d6f2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T20:25:49.034 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:49 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:49.034+0000 7f4c9d6f2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T20:25:49.326 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:49 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:49.125+0000 7f4c9d6f2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T20:25:49.326 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:49 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:49.325+0000 7f4c9d6f2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T20:25:49.713 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:49 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:49.513+0000 7f4c9d6f2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T20:25:50.207 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:50 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:50.119+0000 7f4c9d6f2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T20:25:50.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:50 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/183895321' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-03T20:25:50.463 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:50 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:50.359+0000 7f4c9d6f2200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T20:25:50.898 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout 2024-05-03T20:25:50.898 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout { 2024-05-03T20:25:50.898 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "fsid": "1cd058ee-098b-11ef-bc97-c7b262605968", 2024-05-03T20:25:50.898 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "health": { 2024-05-03T20:25:50.898 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-03T20:25:50.898 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-03T20:25:50.898 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-03T20:25:50.898 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:50.898 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-03T20:25:50.898 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-03T20:25:50.898 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout 0 2024-05-03T20:25:50.898 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout ], 2024-05-03T20:25:50.899 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-03T20:25:50.899 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "a" 2024-05-03T20:25:50.899 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout ], 2024-05-03T20:25:50.899 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-03T20:25:50.899 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-03T20:25:50.899 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T20:25:50.899 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-03T20:25:50.899 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-03T20:25:50.899 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:50.899 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-03T20:25:50.899 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T20:25:50.899 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-03T20:25:50.899 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-03T20:25:50.899 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-03T20:25:50.907 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-03T20:25:50.907 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-03T20:25:50.907 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-03T20:25:50.907 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:50.907 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-03T20:25:50.907 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-03T20:25:50.908 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-03T20:25:50.908 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-03T20:25:50.908 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-03T20:25:50.908 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-03T20:25:50.908 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-03T20:25:50.908 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-03T20:25:50.908 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-03T20:25:50.908 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:50.908 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-03T20:25:50.908 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T20:25:50.908 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-03T20:25:50.908 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-03T20:25:50.908 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:50.908 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-03T20:25:50.908 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-03T20:25:50.908 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-03T20:25:50.909 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-03T20:25:50.909 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-03T20:25:50.909 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-03T20:25:50.909 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "restful" 2024-05-03T20:25:50.909 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout ], 2024-05-03T20:25:50.909 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T20:25:50.909 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:50.909 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-03T20:25:50.909 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T20:25:50.909 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "modified": "2024-05-03T20:25:39.428264+0000", 2024-05-03T20:25:50.909 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T20:25:50.909 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:50.909 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-03T20:25:50.909 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout } 2024-05-03T20:25:50.909 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.215+0000 7f3396160700 1 Processor -- start 2024-05-03T20:25:50.909 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.215+0000 7f3396160700 1 -- start start 2024-05-03T20:25:50.910 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.215+0000 7f3396160700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3390107880 0x7f3390107ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:25:50.910 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.215+0000 7f3396160700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3390076d00 con 0x7f3390107880 2024-05-03T20:25:50.910 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.215+0000 7f338f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3390107880 0x7f3390107ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:25:50.910 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.215+0000 7f338f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3390107880 0x7f3390107ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54416/0 (socket says 172.21.15.181:54416) 2024-05-03T20:25:50.910 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.215+0000 7f338f7fe700 1 -- 172.21.15.181:0/3438529166 learned_addr learned my addr 172.21.15.181:0/3438529166 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:25:50.910 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.216+0000 7f338f7fe700 1 -- 172.21.15.181:0/3438529166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33901081e0 con 0x7f3390107880 2024-05-03T20:25:50.910 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.216+0000 7f338f7fe700 1 --2- 172.21.15.181:0/3438529166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3390107880 0x7f3390107ca0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f33840098a0 tx=0x7f3384009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cdbe671de80a7b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:25:50.910 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.216+0000 7f338e7fc700 1 -- 172.21.15.181:0/3438529166 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3384004160 con 0x7f3390107880 2024-05-03T20:25:50.910 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.216+0000 7f338e7fc700 1 -- 172.21.15.181:0/3438529166 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3384004300 con 0x7f3390107880 2024-05-03T20:25:50.910 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.216+0000 7f338e7fc700 1 -- 172.21.15.181:0/3438529166 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3384011af0 con 0x7f3390107880 2024-05-03T20:25:50.910 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.217+0000 7f3396160700 1 -- 172.21.15.181:0/3438529166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3390107880 msgr2=0x7f3390107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:25:50.910 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.217+0000 7f3396160700 1 --2- 172.21.15.181:0/3438529166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3390107880 0x7f3390107ca0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f33840098a0 tx=0x7f3384009bb0 comp rx=0 tx=0).stop 2024-05-03T20:25:50.910 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.217+0000 7f3396160700 1 -- 172.21.15.181:0/3438529166 shutdown_connections 2024-05-03T20:25:50.910 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.217+0000 7f3396160700 1 --2- 172.21.15.181:0/3438529166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3390107880 0x7f3390107ca0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:25:50.910 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.217+0000 7f3396160700 1 -- 172.21.15.181:0/3438529166 >> 172.21.15.181:0/3438529166 conn(0x7f3390103480 msgr2=0x7f33901058a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:25:50.911 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.217+0000 7f3396160700 1 -- 172.21.15.181:0/3438529166 shutdown_connections 2024-05-03T20:25:50.911 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.217+0000 7f3396160700 1 -- 172.21.15.181:0/3438529166 wait complete. 2024-05-03T20:25:50.911 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.217+0000 7f3396160700 1 Processor -- start 2024-05-03T20:25:50.911 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.218+0000 7f3396160700 1 -- start start 2024-05-03T20:25:50.911 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.218+0000 7f3396160700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3390107880 0x7f339019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:25:50.911 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.218+0000 7f3396160700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f33901091a0 con 0x7f3390107880 2024-05-03T20:25:50.911 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.218+0000 7f338f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3390107880 0x7f339019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:25:50.911 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.218+0000 7f338f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3390107880 0x7f339019efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54422/0 (socket says 172.21.15.181:54422) 2024-05-03T20:25:50.911 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.218+0000 7f338f7fe700 1 -- 172.21.15.181:0/183895321 learned_addr learned my addr 172.21.15.181:0/183895321 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:25:50.911 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.218+0000 7f338f7fe700 1 -- 172.21.15.181:0/183895321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f339019f4f0 con 0x7f3390107880 2024-05-03T20:25:50.911 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.218+0000 7f338f7fe700 1 --2- 172.21.15.181:0/183895321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3390107880 0x7f339019efb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3384009fb0 tx=0x7f3384011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:25:50.911 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.218+0000 7f338cff9700 1 -- 172.21.15.181:0/183895321 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33840036a0 con 0x7f3390107880 2024-05-03T20:25:50.911 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.218+0000 7f3396160700 1 -- 172.21.15.181:0/183895321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f339019f780 con 0x7f3390107880 2024-05-03T20:25:50.911 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.219+0000 7f338cff9700 1 -- 172.21.15.181:0/183895321 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f338401a430 con 0x7f3390107880 2024-05-03T20:25:50.912 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.219+0000 7f338cff9700 1 -- 172.21.15.181:0/183895321 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3384019550 con 0x7f3390107880 2024-05-03T20:25:50.912 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.219+0000 7f3396160700 1 -- 172.21.15.181:0/183895321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33901a2470 con 0x7f3390107880 2024-05-03T20:25:50.912 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.219+0000 7f338cff9700 1 -- 172.21.15.181:0/183895321 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3384019be0 con 0x7f3390107880 2024-05-03T20:25:50.912 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.219+0000 7f338cff9700 1 -- 172.21.15.181:0/183895321 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3384024730 con 0x7f3390107880 2024-05-03T20:25:50.912 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.219+0000 7f3396160700 1 -- 172.21.15.181:0/183895321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3390064b00 con 0x7f3390107880 2024-05-03T20:25:50.912 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.223+0000 7f338cff9700 1 -- 172.21.15.181:0/183895321 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3384043e50 con 0x7f3390107880 2024-05-03T20:25:50.912 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.297+0000 7f3396160700 1 -- 172.21.15.181:0/183895321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f339002e710 con 0x7f3390107880 2024-05-03T20:25:50.912 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.298+0000 7f338cff9700 1 -- 172.21.15.181:0/183895321 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f3384032030 con 0x7f3390107880 2024-05-03T20:25:50.912 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.299+0000 7f3396160700 1 -- 172.21.15.181:0/183895321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3390107880 msgr2=0x7f339019efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:25:50.912 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.299+0000 7f3396160700 1 --2- 172.21.15.181:0/183895321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3390107880 0x7f339019efb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3384009fb0 tx=0x7f3384011940 comp rx=0 tx=0).stop 2024-05-03T20:25:50.912 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.299+0000 7f3396160700 1 -- 172.21.15.181:0/183895321 shutdown_connections 2024-05-03T20:25:50.912 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.299+0000 7f3396160700 1 --2- 172.21.15.181:0/183895321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3390107880 0x7f339019efb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:25:50.912 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.299+0000 7f3396160700 1 -- 172.21.15.181:0/183895321 >> 172.21.15.181:0/183895321 conn(0x7f3390103480 msgr2=0x7f3390195c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:25:50.912 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.299+0000 7f3396160700 1 -- 172.21.15.181:0/183895321 shutdown_connections 2024-05-03T20:25:50.912 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:50.299+0000 7f3396160700 1 -- 172.21.15.181:0/183895321 wait complete. 2024-05-03T20:25:50.913 INFO:teuthology.orchestra.run.smithi181.stdout:mgr not available, waiting (2/15)... 2024-05-03T20:25:51.213 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:50 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:50.788+0000 7f4c9d6f2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T20:25:51.213 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:50 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:50.886+0000 7f4c9d6f2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T20:25:51.613 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:51 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:51.260+0000 7f4c9d6f2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T20:25:51.613 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:51 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:51.613+0000 7f4c9d6f2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T20:25:51.946 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:51 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:51.705+0000 7f4c9d6f2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T20:25:52.213 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:51 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:51.946+0000 7f4c9d6f2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T20:25:52.213 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:52 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:52.077+0000 7f4c9d6f2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T20:25:52.713 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:52 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:52.385+0000 7f4c9d6f2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T20:25:52.713 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:52 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:52.499+0000 7f4c9d6f2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T20:25:52.714 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:52 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:52.590+0000 7f4c9d6f2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T20:25:53.312 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:53 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:53.199+0000 7f4c9d6f2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T20:25:53.713 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:53 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:53.425+0000 7f4c9d6f2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T20:25:53.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:53 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3976777487' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-03T20:25:53.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:53 smithi181 ceph-mon[29801]: Activating manager daemon x 2024-05-03T20:25:53.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:53 smithi181 ceph-mon[29801]: mgrmap e2: x(active, starting, since 0.00396746s) 2024-05-03T20:25:53.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:53 smithi181 ceph-mon[29801]: from='mgr.14100 172.21.15.181:0/199192957' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-03T20:25:53.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:53 smithi181 ceph-mon[29801]: from='mgr.14100 172.21.15.181:0/199192957' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-03T20:25:53.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:53 smithi181 ceph-mon[29801]: from='mgr.14100 172.21.15.181:0/199192957' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-03T20:25:53.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:53 smithi181 ceph-mon[29801]: from='mgr.14100 172.21.15.181:0/199192957' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T20:25:53.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:53 smithi181 ceph-mon[29801]: from='mgr.14100 172.21.15.181:0/199192957' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-03T20:25:53.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:53 smithi181 ceph-mon[29801]: Manager daemon x is now available 2024-05-03T20:25:53.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:53 smithi181 ceph-mon[29801]: from='mgr.14100 172.21.15.181:0/199192957' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-03T20:25:53.715 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:53 smithi181 ceph-mon[29801]: from='mgr.14100 172.21.15.181:0/199192957' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-03T20:25:54.022 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout 2024-05-03T20:25:54.022 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout { 2024-05-03T20:25:54.022 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "fsid": "1cd058ee-098b-11ef-bc97-c7b262605968", 2024-05-03T20:25:54.022 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "health": { 2024-05-03T20:25:54.023 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-03T20:25:54.023 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-03T20:25:54.023 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-03T20:25:54.023 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:54.023 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-03T20:25:54.023 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-03T20:25:54.023 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout 0 2024-05-03T20:25:54.023 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout ], 2024-05-03T20:25:54.023 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-03T20:25:54.023 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "a" 2024-05-03T20:25:54.023 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout ], 2024-05-03T20:25:54.023 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-05-03T20:25:54.023 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-03T20:25:54.023 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T20:25:54.023 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-03T20:25:54.024 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-03T20:25:54.024 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:54.024 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-03T20:25:54.024 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T20:25:54.024 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-03T20:25:54.024 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-03T20:25:54.024 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-03T20:25:54.024 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-03T20:25:54.024 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-03T20:25:54.024 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-03T20:25:54.024 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:54.024 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-03T20:25:54.024 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-03T20:25:54.024 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-03T20:25:54.024 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-03T20:25:54.024 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-03T20:25:54.025 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-03T20:25:54.025 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-03T20:25:54.025 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-03T20:25:54.025 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-03T20:25:54.025 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:54.025 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-03T20:25:54.025 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T20:25:54.025 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-03T20:25:54.025 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-03T20:25:54.025 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:54.025 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-03T20:25:54.025 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-03T20:25:54.025 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-03T20:25:54.025 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-03T20:25:54.026 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-03T20:25:54.026 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-03T20:25:54.026 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "restful" 2024-05-03T20:25:54.026 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout ], 2024-05-03T20:25:54.026 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T20:25:54.026 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:54.026 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-03T20:25:54.026 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T20:25:54.026 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "modified": "2024-05-03T20:25:39.428264+0000", 2024-05-03T20:25:54.026 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T20:25:54.026 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:54.026 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-03T20:25:54.026 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout } 2024-05-03T20:25:54.026 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.327+0000 7f6d4b6c4700 1 Processor -- start 2024-05-03T20:25:54.027 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.327+0000 7f6d4b6c4700 1 -- start start 2024-05-03T20:25:54.027 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.327+0000 7f6d4b6c4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d44108f80 0x7f6d441093a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:25:54.027 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.327+0000 7f6d4b6c4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6d44109970 con 0x7f6d44108f80 2024-05-03T20:25:54.027 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.328+0000 7f6d49460700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d44108f80 0x7f6d441093a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:25:54.027 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.328+0000 7f6d49460700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d44108f80 0x7f6d441093a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54428/0 (socket says 172.21.15.181:54428) 2024-05-03T20:25:54.027 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.328+0000 7f6d49460700 1 -- 172.21.15.181:0/2054258481 learned_addr learned my addr 172.21.15.181:0/2054258481 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:25:54.027 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.328+0000 7f6d49460700 1 -- 172.21.15.181:0/2054258481 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4410a180 con 0x7f6d44108f80 2024-05-03T20:25:54.027 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.328+0000 7f6d49460700 1 --2- 172.21.15.181:0/2054258481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d44108f80 0x7f6d441093a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6d400098a0 tx=0x7f6d40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d706e6e6b02c6e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:25:54.027 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.328+0000 7f6d3bfff700 1 -- 172.21.15.181:0/2054258481 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d40004160 con 0x7f6d44108f80 2024-05-03T20:25:54.027 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.328+0000 7f6d3bfff700 1 -- 172.21.15.181:0/2054258481 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6d40004300 con 0x7f6d44108f80 2024-05-03T20:25:54.027 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.328+0000 7f6d3bfff700 1 -- 172.21.15.181:0/2054258481 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d40011af0 con 0x7f6d44108f80 2024-05-03T20:25:54.027 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.329+0000 7f6d4b6c4700 1 -- 172.21.15.181:0/2054258481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d44108f80 msgr2=0x7f6d441093a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:25:54.027 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.329+0000 7f6d4b6c4700 1 --2- 172.21.15.181:0/2054258481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d44108f80 0x7f6d441093a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6d400098a0 tx=0x7f6d40009bb0 comp rx=0 tx=0).stop 2024-05-03T20:25:54.027 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.329+0000 7f6d4b6c4700 1 -- 172.21.15.181:0/2054258481 shutdown_connections 2024-05-03T20:25:54.028 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.329+0000 7f6d4b6c4700 1 --2- 172.21.15.181:0/2054258481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d44108f80 0x7f6d441093a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:25:54.028 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.329+0000 7f6d4b6c4700 1 -- 172.21.15.181:0/2054258481 >> 172.21.15.181:0/2054258481 conn(0x7f6d44104540 msgr2=0x7f6d44106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:25:54.028 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.329+0000 7f6d4b6c4700 1 -- 172.21.15.181:0/2054258481 shutdown_connections 2024-05-03T20:25:54.028 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.330+0000 7f6d4b6c4700 1 -- 172.21.15.181:0/2054258481 wait complete. 2024-05-03T20:25:54.028 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.330+0000 7f6d4b6c4700 1 Processor -- start 2024-05-03T20:25:54.028 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.330+0000 7f6d4b6c4700 1 -- start start 2024-05-03T20:25:54.028 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.330+0000 7f6d4b6c4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d44108f80 0x7f6d4419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:25:54.028 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.330+0000 7f6d4b6c4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4410d3b0 con 0x7f6d44108f80 2024-05-03T20:25:54.028 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.331+0000 7f6d49460700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d44108f80 0x7f6d4419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:25:54.028 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.331+0000 7f6d49460700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d44108f80 0x7f6d4419ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54438/0 (socket says 172.21.15.181:54438) 2024-05-03T20:25:54.028 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.331+0000 7f6d49460700 1 -- 172.21.15.181:0/3976777487 learned_addr learned my addr 172.21.15.181:0/3976777487 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:25:54.028 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.331+0000 7f6d49460700 1 -- 172.21.15.181:0/3976777487 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4419d360 con 0x7f6d44108f80 2024-05-03T20:25:54.028 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.331+0000 7f6d49460700 1 --2- 172.21.15.181:0/3976777487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d44108f80 0x7f6d4419ce20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009fb0 tx=0x7f6d40011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:25:54.028 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.331+0000 7f6d3a7fc700 1 -- 172.21.15.181:0/3976777487 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d400036a0 con 0x7f6d44108f80 2024-05-03T20:25:54.028 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.331+0000 7f6d3a7fc700 1 -- 172.21.15.181:0/3976777487 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6d4001a430 con 0x7f6d44108f80 2024-05-03T20:25:54.028 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.331+0000 7f6d4b6c4700 1 -- 172.21.15.181:0/3976777487 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d4419d5f0 con 0x7f6d44108f80 2024-05-03T20:25:54.029 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.331+0000 7f6d3a7fc700 1 -- 172.21.15.181:0/3976777487 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d40019550 con 0x7f6d44108f80 2024-05-03T20:25:54.029 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.331+0000 7f6d4b6c4700 1 -- 172.21.15.181:0/3976777487 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d441a02e0 con 0x7f6d44108f80 2024-05-03T20:25:54.029 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.332+0000 7f6d3a7fc700 1 -- 172.21.15.181:0/3976777487 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6d40019b90 con 0x7f6d44108f80 2024-05-03T20:25:54.029 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.332+0000 7f6d3a7fc700 1 -- 172.21.15.181:0/3976777487 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6d40024730 con 0x7f6d44108f80 2024-05-03T20:25:54.029 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.332+0000 7f6d4b6c4700 1 -- 172.21.15.181:0/3976777487 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d4410d530 con 0x7f6d44108f80 2024-05-03T20:25:54.029 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.334+0000 7f6d3a7fc700 1 -- 172.21.15.181:0/3976777487 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6d40043e50 con 0x7f6d44108f80 2024-05-03T20:25:54.029 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.409+0000 7f6d4b6c4700 1 -- 172.21.15.181:0/3976777487 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6d441a0810 con 0x7f6d44108f80 2024-05-03T20:25:54.029 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.409+0000 7f6d3a7fc700 1 -- 172.21.15.181:0/3976777487 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f6d40032020 con 0x7f6d44108f80 2024-05-03T20:25:54.029 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.410+0000 7f6d4b6c4700 1 -- 172.21.15.181:0/3976777487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d44108f80 msgr2=0x7f6d4419ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:25:54.029 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.410+0000 7f6d4b6c4700 1 --2- 172.21.15.181:0/3976777487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d44108f80 0x7f6d4419ce20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009fb0 tx=0x7f6d40011940 comp rx=0 tx=0).stop 2024-05-03T20:25:54.029 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.410+0000 7f6d4b6c4700 1 -- 172.21.15.181:0/3976777487 shutdown_connections 2024-05-03T20:25:54.029 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.410+0000 7f6d4b6c4700 1 --2- 172.21.15.181:0/3976777487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d44108f80 0x7f6d4419ce20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:25:54.029 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.410+0000 7f6d4b6c4700 1 -- 172.21.15.181:0/3976777487 >> 172.21.15.181:0/3976777487 conn(0x7f6d44104540 msgr2=0x7f6d44104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:25:54.029 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.411+0000 7f6d4b6c4700 1 -- 172.21.15.181:0/3976777487 shutdown_connections 2024-05-03T20:25:54.029 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:53.411+0000 7f6d4b6c4700 1 -- 172.21.15.181:0/3976777487 wait complete. 2024-05-03T20:25:54.029 INFO:teuthology.orchestra.run.smithi181.stdout:mgr not available, waiting (3/15)... 2024-05-03T20:25:54.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:54 smithi181 ceph-mon[29801]: from='mgr.14100 172.21.15.181:0/199192957' entity='mgr.x' 2024-05-03T20:25:54.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:54 smithi181 ceph-mon[29801]: from='mgr.14100 172.21.15.181:0/199192957' entity='mgr.x' 2024-05-03T20:25:54.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:54 smithi181 ceph-mon[29801]: from='mgr.14100 172.21.15.181:0/199192957' entity='mgr.x' 2024-05-03T20:25:54.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:54 smithi181 ceph-mon[29801]: mgrmap e3: x(active, since 1.00718s) 2024-05-03T20:25:56.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:56 smithi181 ceph-mon[29801]: mgrmap e4: x(active, since 2s) 2024-05-03T20:25:57.352 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout 2024-05-03T20:25:57.353 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout { 2024-05-03T20:25:57.353 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "fsid": "1cd058ee-098b-11ef-bc97-c7b262605968", 2024-05-03T20:25:57.353 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "health": { 2024-05-03T20:25:57.353 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-03T20:25:57.353 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-03T20:25:57.353 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-03T20:25:57.353 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:57.353 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-03T20:25:57.353 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-03T20:25:57.353 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout 0 2024-05-03T20:25:57.353 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout ], 2024-05-03T20:25:57.353 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-03T20:25:57.353 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "a" 2024-05-03T20:25:57.353 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout ], 2024-05-03T20:25:57.353 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-05-03T20:25:57.354 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-03T20:25:57.354 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T20:25:57.354 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-03T20:25:57.354 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-03T20:25:57.354 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:57.354 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-03T20:25:57.354 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T20:25:57.354 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-03T20:25:57.354 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-03T20:25:57.354 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-03T20:25:57.363 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-03T20:25:57.363 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-03T20:25:57.363 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-03T20:25:57.363 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:57.363 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-03T20:25:57.363 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-03T20:25:57.363 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-03T20:25:57.363 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-03T20:25:57.363 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-03T20:25:57.363 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-03T20:25:57.363 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-03T20:25:57.364 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-03T20:25:57.364 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-03T20:25:57.364 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:57.364 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-03T20:25:57.364 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T20:25:57.364 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-03T20:25:57.364 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-03T20:25:57.364 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:57.364 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-03T20:25:57.364 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-03T20:25:57.364 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-03T20:25:57.364 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-03T20:25:57.364 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-03T20:25:57.364 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-03T20:25:57.364 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "restful" 2024-05-03T20:25:57.364 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout ], 2024-05-03T20:25:57.365 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T20:25:57.365 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:57.365 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-03T20:25:57.365 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T20:25:57.365 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "modified": "2024-05-03T20:25:39.428264+0000", 2024-05-03T20:25:57.365 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T20:25:57.365 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout }, 2024-05-03T20:25:57.365 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-03T20:25:57.365 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout } 2024-05-03T20:25:57.365 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.438+0000 7fb36b8a5700 1 Processor -- start 2024-05-03T20:25:57.365 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.438+0000 7fb36b8a5700 1 -- start start 2024-05-03T20:25:57.365 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.438+0000 7fb36b8a5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb364108380 0x7fb3641087a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:25:57.365 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.438+0000 7fb36b8a5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb364108d70 con 0x7fb364108380 2024-05-03T20:25:57.365 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.438+0000 7fb369641700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb364108380 0x7fb3641087a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:25:57.365 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.438+0000 7fb369641700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb364108380 0x7fb3641087a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60464/0 (socket says 172.21.15.181:60464) 2024-05-03T20:25:57.365 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.438+0000 7fb369641700 1 -- 172.21.15.181:0/2533814709 learned_addr learned my addr 172.21.15.181:0/2533814709 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:25:57.366 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.439+0000 7fb369641700 1 -- 172.21.15.181:0/2533814709 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3641090a0 con 0x7fb364108380 2024-05-03T20:25:57.366 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.439+0000 7fb369641700 1 --2- 172.21.15.181:0/2533814709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb364108380 0x7fb3641087a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb3600098a0 tx=0x7fb360009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60218f32f6298c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:25:57.366 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.439+0000 7fb35bfff700 1 -- 172.21.15.181:0/2533814709 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb360004160 con 0x7fb364108380 2024-05-03T20:25:57.366 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.439+0000 7fb35bfff700 1 -- 172.21.15.181:0/2533814709 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb360004300 con 0x7fb364108380 2024-05-03T20:25:57.366 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.439+0000 7fb35bfff700 1 -- 172.21.15.181:0/2533814709 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb360011af0 con 0x7fb364108380 2024-05-03T20:25:57.366 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.439+0000 7fb36b8a5700 1 -- 172.21.15.181:0/2533814709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb364108380 msgr2=0x7fb3641087a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:25:57.366 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.440+0000 7fb36b8a5700 1 --2- 172.21.15.181:0/2533814709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb364108380 0x7fb3641087a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb3600098a0 tx=0x7fb360009bb0 comp rx=0 tx=0).stop 2024-05-03T20:25:57.366 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.440+0000 7fb36b8a5700 1 -- 172.21.15.181:0/2533814709 shutdown_connections 2024-05-03T20:25:57.366 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.440+0000 7fb36b8a5700 1 --2- 172.21.15.181:0/2533814709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb364108380 0x7fb3641087a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:25:57.366 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.440+0000 7fb36b8a5700 1 -- 172.21.15.181:0/2533814709 >> 172.21.15.181:0/2533814709 conn(0x7fb364102c40 msgr2=0x7fb364105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:25:57.366 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.440+0000 7fb36b8a5700 1 -- 172.21.15.181:0/2533814709 shutdown_connections 2024-05-03T20:25:57.366 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.440+0000 7fb36b8a5700 1 -- 172.21.15.181:0/2533814709 wait complete. 2024-05-03T20:25:57.366 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.440+0000 7fb36b8a5700 1 Processor -- start 2024-05-03T20:25:57.366 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.440+0000 7fb36b8a5700 1 -- start start 2024-05-03T20:25:57.367 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.440+0000 7fb36b8a5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb364108380 0x7fb36419bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:25:57.367 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.440+0000 7fb36b8a5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb36410c2d0 con 0x7fb364108380 2024-05-03T20:25:57.367 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.441+0000 7fb369641700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb364108380 0x7fb36419bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:25:57.367 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.441+0000 7fb369641700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb364108380 0x7fb36419bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60474/0 (socket says 172.21.15.181:60474) 2024-05-03T20:25:57.367 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.441+0000 7fb369641700 1 -- 172.21.15.181:0/3289122000 learned_addr learned my addr 172.21.15.181:0/3289122000 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:25:57.367 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.441+0000 7fb369641700 1 -- 172.21.15.181:0/3289122000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb36419c230 con 0x7fb364108380 2024-05-03T20:25:57.367 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.441+0000 7fb369641700 1 --2- 172.21.15.181:0/3289122000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb364108380 0x7fb36419bcf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb360009fb0 tx=0x7fb360011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:25:57.367 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.441+0000 7fb35a7fc700 1 -- 172.21.15.181:0/3289122000 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3600036a0 con 0x7fb364108380 2024-05-03T20:25:57.367 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.441+0000 7fb36b8a5700 1 -- 172.21.15.181:0/3289122000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb36419c4c0 con 0x7fb364108380 2024-05-03T20:25:57.367 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.441+0000 7fb35a7fc700 1 -- 172.21.15.181:0/3289122000 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb36001a430 con 0x7fb364108380 2024-05-03T20:25:57.367 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.441+0000 7fb35a7fc700 1 -- 172.21.15.181:0/3289122000 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb360019550 con 0x7fb364108380 2024-05-03T20:25:57.367 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.441+0000 7fb36b8a5700 1 -- 172.21.15.181:0/3289122000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb36419f1a0 con 0x7fb364108380 2024-05-03T20:25:57.367 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.442+0000 7fb35a7fc700 1 -- 172.21.15.181:0/3289122000 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fb36001a8e0 con 0x7fb364108380 2024-05-03T20:25:57.367 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.442+0000 7fb36b8a5700 1 -- 172.21.15.181:0/3289122000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb36410c450 con 0x7fb364108380 2024-05-03T20:25:57.367 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.442+0000 7fb35a7fc700 1 --2- 172.21.15.181:0/3289122000 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7fb35003e490 0x7fb350040950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:25:57.368 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.442+0000 7fb35a7fc700 1 -- 172.21.15.181:0/3289122000 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb3600537e0 con 0x7fb364108380 2024-05-03T20:25:57.368 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.442+0000 7fb368e40700 1 --2- 172.21.15.181:0/3289122000 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7fb35003e490 0x7fb350040950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:25:57.368 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.443+0000 7fb368e40700 1 --2- 172.21.15.181:0/3289122000 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7fb35003e490 0x7fb350040950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb354006fd0 tx=0x7fb354006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:25:57.368 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.448+0000 7fb35a7fc700 1 -- 172.21.15.181:0/3289122000 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb360008040 con 0x7fb364108380 2024-05-03T20:25:57.368 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.738+0000 7fb36b8a5700 1 -- 172.21.15.181:0/3289122000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb36419f7f0 con 0x7fb364108380 2024-05-03T20:25:57.368 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.738+0000 7fb35a7fc700 1 -- 172.21.15.181:0/3289122000 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fb36000b3e0 con 0x7fb364108380 2024-05-03T20:25:57.368 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.740+0000 7fb36b8a5700 1 -- 172.21.15.181:0/3289122000 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7fb35003e490 msgr2=0x7fb350040950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:25:57.368 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.740+0000 7fb36b8a5700 1 --2- 172.21.15.181:0/3289122000 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7fb35003e490 0x7fb350040950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb354006fd0 tx=0x7fb354006e40 comp rx=0 tx=0).stop 2024-05-03T20:25:57.368 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.740+0000 7fb36b8a5700 1 -- 172.21.15.181:0/3289122000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb364108380 msgr2=0x7fb36419bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:25:57.368 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.740+0000 7fb36b8a5700 1 --2- 172.21.15.181:0/3289122000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb364108380 0x7fb36419bcf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb360009fb0 tx=0x7fb360011940 comp rx=0 tx=0).stop 2024-05-03T20:25:57.368 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.740+0000 7fb36b8a5700 1 -- 172.21.15.181:0/3289122000 shutdown_connections 2024-05-03T20:25:57.368 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.740+0000 7fb36b8a5700 1 --2- 172.21.15.181:0/3289122000 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7fb35003e490 0x7fb350040950 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:25:57.368 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.740+0000 7fb36b8a5700 1 --2- 172.21.15.181:0/3289122000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb364108380 0x7fb36419bcf0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:25:57.368 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.740+0000 7fb36b8a5700 1 -- 172.21.15.181:0/3289122000 >> 172.21.15.181:0/3289122000 conn(0x7fb364102c40 msgr2=0x7fb364103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:25:57.368 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.741+0000 7fb36b8a5700 1 -- 172.21.15.181:0/3289122000 shutdown_connections 2024-05-03T20:25:57.368 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:56.741+0000 7fb36b8a5700 1 -- 172.21.15.181:0/3289122000 wait complete. 2024-05-03T20:25:57.369 INFO:teuthology.orchestra.run.smithi181.stdout:mgr is available 2024-05-03T20:25:57.650 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:57 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3289122000' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-03T20:25:58.545 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:58 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/2801389566' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-03T20:25:58.556 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout 2024-05-03T20:25:58.556 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout [global] 2024-05-03T20:25:58.556 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout fsid = 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:25:58.556 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-03T20:25:58.556 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-03T20:25:58.556 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-03T20:25:58.556 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-03T20:25:58.556 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-03T20:25:58.556 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout 2024-05-03T20:25:58.556 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-03T20:25:58.556 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-03T20:25:58.557 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout 2024-05-03T20:25:58.557 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout [osd] 2024-05-03T20:25:58.557 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-05-03T20:25:58.557 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-03T20:25:58.557 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-03T20:25:58.557 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-03T20:25:58.557 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-03T20:25:58.557 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-03T20:25:58.557 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.750+0000 7fef0a939700 1 Processor -- start 2024-05-03T20:25:58.557 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.750+0000 7fef0a939700 1 -- start start 2024-05-03T20:25:58.557 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.750+0000 7fef0a939700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef04109410 0x7fef04109830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:25:58.557 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.750+0000 7fef0a939700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fef04109e00 con 0x7fef04109410 2024-05-03T20:25:58.557 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.750+0000 7fef03fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef04109410 0x7fef04109830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:25:58.557 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.751+0000 7fef03fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef04109410 0x7fef04109830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60480/0 (socket says 172.21.15.181:60480) 2024-05-03T20:25:58.557 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.751+0000 7fef03fff700 1 -- 172.21.15.181:0/1556107001 learned_addr learned my addr 172.21.15.181:0/1556107001 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:25:58.558 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.751+0000 7fef03fff700 1 -- 172.21.15.181:0/1556107001 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef04108070 con 0x7fef04109410 2024-05-03T20:25:58.558 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.751+0000 7fef03fff700 1 --2- 172.21.15.181:0/1556107001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef04109410 0x7fef04109830 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7feef800bb40 tx=0x7feef800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b43544e159c50e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:25:58.558 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.751+0000 7fef02ffd700 1 -- 172.21.15.181:0/1556107001 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feef800dbe0 con 0x7fef04109410 2024-05-03T20:25:58.558 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.751+0000 7fef02ffd700 1 -- 172.21.15.181:0/1556107001 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7feef80044c0 con 0x7fef04109410 2024-05-03T20:25:58.558 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.751+0000 7fef02ffd700 1 -- 172.21.15.181:0/1556107001 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feef8010470 con 0x7fef04109410 2024-05-03T20:25:58.558 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.752+0000 7fef0a939700 1 -- 172.21.15.181:0/1556107001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef04109410 msgr2=0x7fef04109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:25:58.558 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.752+0000 7fef0a939700 1 --2- 172.21.15.181:0/1556107001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef04109410 0x7fef04109830 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7feef800bb40 tx=0x7feef800be50 comp rx=0 tx=0).stop 2024-05-03T20:25:58.558 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.752+0000 7fef0a939700 1 -- 172.21.15.181:0/1556107001 shutdown_connections 2024-05-03T20:25:58.558 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.752+0000 7fef0a939700 1 --2- 172.21.15.181:0/1556107001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef04109410 0x7fef04109830 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:25:58.558 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.752+0000 7fef0a939700 1 -- 172.21.15.181:0/1556107001 >> 172.21.15.181:0/1556107001 conn(0x7fef04102c00 msgr2=0x7fef04105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:25:58.558 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.752+0000 7fef0a939700 1 -- 172.21.15.181:0/1556107001 shutdown_connections 2024-05-03T20:25:58.558 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.752+0000 7fef0a939700 1 -- 172.21.15.181:0/1556107001 wait complete. 2024-05-03T20:25:58.558 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.752+0000 7fef0a939700 1 Processor -- start 2024-05-03T20:25:58.558 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.752+0000 7fef0a939700 1 -- start start 2024-05-03T20:25:58.558 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.753+0000 7fef0a939700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef04109410 0x7fef0419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:25:58.558 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.753+0000 7fef0a939700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fef04106f60 con 0x7fef04109410 2024-05-03T20:25:58.559 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.753+0000 7fef03fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef04109410 0x7fef0419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:25:58.559 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.753+0000 7fef03fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef04109410 0x7fef0419ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60494/0 (socket says 172.21.15.181:60494) 2024-05-03T20:25:58.559 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.753+0000 7fef03fff700 1 -- 172.21.15.181:0/2801389566 learned_addr learned my addr 172.21.15.181:0/2801389566 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:25:58.559 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.753+0000 7fef03fff700 1 -- 172.21.15.181:0/2801389566 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0419b0c0 con 0x7fef04109410 2024-05-03T20:25:58.559 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.753+0000 7fef03fff700 1 --2- 172.21.15.181:0/2801389566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef04109410 0x7fef0419ab80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7feef800b340 tx=0x7feef8003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:25:58.559 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.753+0000 7fef017fa700 1 -- 172.21.15.181:0/2801389566 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feef8027070 con 0x7fef04109410 2024-05-03T20:25:58.559 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.753+0000 7fef017fa700 1 -- 172.21.15.181:0/2801389566 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7feef8010bb0 con 0x7fef04109410 2024-05-03T20:25:58.559 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.753+0000 7fef0a939700 1 -- 172.21.15.181:0/2801389566 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef0419b350 con 0x7fef04109410 2024-05-03T20:25:58.559 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.753+0000 7fef017fa700 1 -- 172.21.15.181:0/2801389566 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feef8022b30 con 0x7fef04109410 2024-05-03T20:25:58.559 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.754+0000 7fef0a939700 1 -- 172.21.15.181:0/2801389566 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef0419dff0 con 0x7fef04109410 2024-05-03T20:25:58.559 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.754+0000 7fef017fa700 1 -- 172.21.15.181:0/2801389566 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7feef802a070 con 0x7fef04109410 2024-05-03T20:25:58.559 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.755+0000 7fef017fa700 1 --2- 172.21.15.181:0/2801389566 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7feeec03e4e0 0x7feeec0409a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:25:58.559 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.755+0000 7fef017fa700 1 -- 172.21.15.181:0/2801389566 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7feef801f070 con 0x7fef04109410 2024-05-03T20:25:58.559 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.755+0000 7fef037fe700 1 --2- 172.21.15.181:0/2801389566 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7feeec03e4e0 0x7feeec0409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:25:58.560 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.755+0000 7fef0a939700 1 -- 172.21.15.181:0/2801389566 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef04064b00 con 0x7fef04109410 2024-05-03T20:25:58.560 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.755+0000 7fef037fe700 1 --2- 172.21.15.181:0/2801389566 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7feeec03e4e0 0x7feeec0409a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7feef4006fd0 tx=0x7feef4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:25:58.560 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.760+0000 7fef017fa700 1 -- 172.21.15.181:0/2801389566 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7feef8015070 con 0x7fef04109410 2024-05-03T20:25:58.560 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.971+0000 7fef0a939700 1 -- 172.21.15.181:0/2801389566 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fef0419e1c0 con 0x7fef04109410 2024-05-03T20:25:58.560 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.972+0000 7fef017fa700 1 -- 172.21.15.181:0/2801389566 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7feef8012030 con 0x7fef04109410 2024-05-03T20:25:58.560 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.975+0000 7fef0a939700 1 -- 172.21.15.181:0/2801389566 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7feeec03e4e0 msgr2=0x7feeec0409a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:25:58.560 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.975+0000 7fef0a939700 1 --2- 172.21.15.181:0/2801389566 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7feeec03e4e0 0x7feeec0409a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7feef4006fd0 tx=0x7feef4006e40 comp rx=0 tx=0).stop 2024-05-03T20:25:58.560 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.975+0000 7fef0a939700 1 -- 172.21.15.181:0/2801389566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef04109410 msgr2=0x7fef0419ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:25:58.560 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.975+0000 7fef0a939700 1 --2- 172.21.15.181:0/2801389566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef04109410 0x7fef0419ab80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7feef800b340 tx=0x7feef8003dc0 comp rx=0 tx=0).stop 2024-05-03T20:25:58.560 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.975+0000 7fef0a939700 1 -- 172.21.15.181:0/2801389566 shutdown_connections 2024-05-03T20:25:58.560 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.975+0000 7fef0a939700 1 --2- 172.21.15.181:0/2801389566 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7feeec03e4e0 0x7feeec0409a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:25:58.560 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.975+0000 7fef0a939700 1 --2- 172.21.15.181:0/2801389566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef04109410 0x7fef0419ab80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:25:58.560 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.975+0000 7fef0a939700 1 -- 172.21.15.181:0/2801389566 >> 172.21.15.181:0/2801389566 conn(0x7fef04102c00 msgr2=0x7fef041911a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:25:58.560 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.975+0000 7fef0a939700 1 -- 172.21.15.181:0/2801389566 shutdown_connections 2024-05-03T20:25:58.561 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:57.975+0000 7fef0a939700 1 -- 172.21.15.181:0/2801389566 wait complete. 2024-05-03T20:25:58.561 INFO:teuthology.orchestra.run.smithi181.stdout:Enabling cephadm module... 2024-05-03T20:25:59.486 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:25:59 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3715620813' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-03T20:25:59.785 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:59 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: ignoring --setuser ceph since I am not root 2024-05-03T20:25:59.785 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:59 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: ignoring --setgroup ceph since I am not root 2024-05-03T20:25:59.785 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:59 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:59.638+0000 7f0142b8c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T20:26:00.085 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:59 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:59.825+0000 7f0142b8c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T20:26:00.085 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:25:59 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:25:59.927+0000 7f0142b8c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T20:26:00.087 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.958+0000 7f3a6f0f9700 1 Processor -- start 2024-05-03T20:26:00.093 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.958+0000 7f3a6f0f9700 1 -- start start 2024-05-03T20:26:00.093 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.958+0000 7f3a6f0f9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a68108f10 0x7f3a68109350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:00.094 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.958+0000 7f3a6f0f9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3a68109890 con 0x7f3a68108f10 2024-05-03T20:26:00.094 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.958+0000 7f3a6ce95700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a68108f10 0x7f3a68109350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:00.094 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.958+0000 7f3a6ce95700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a68108f10 0x7f3a68109350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60496/0 (socket says 172.21.15.181:60496) 2024-05-03T20:26:00.094 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.958+0000 7f3a6ce95700 1 -- 172.21.15.181:0/651123354 learned_addr learned my addr 172.21.15.181:0/651123354 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:00.094 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.959+0000 7f3a6ce95700 1 -- 172.21.15.181:0/651123354 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6810a0f0 con 0x7f3a68108f10 2024-05-03T20:26:00.094 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.959+0000 7f3a6ce95700 1 --2- 172.21.15.181:0/651123354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a68108f10 0x7f3a68109350 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0098a0 tx=0x7f3a5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48cccc24d7ab0152 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:00.094 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.959+0000 7f3a677fe700 1 -- 172.21.15.181:0/651123354 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a5c004160 con 0x7f3a68108f10 2024-05-03T20:26:00.094 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.959+0000 7f3a677fe700 1 -- 172.21.15.181:0/651123354 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3a5c004300 con 0x7f3a68108f10 2024-05-03T20:26:00.094 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.959+0000 7f3a677fe700 1 -- 172.21.15.181:0/651123354 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a5c011af0 con 0x7f3a68108f10 2024-05-03T20:26:00.094 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.959+0000 7f3a6f0f9700 1 -- 172.21.15.181:0/651123354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a68108f10 msgr2=0x7f3a68109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:00.094 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.959+0000 7f3a6f0f9700 1 --2- 172.21.15.181:0/651123354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a68108f10 0x7f3a68109350 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0098a0 tx=0x7f3a5c009bb0 comp rx=0 tx=0).stop 2024-05-03T20:26:00.094 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.959+0000 7f3a6f0f9700 1 -- 172.21.15.181:0/651123354 shutdown_connections 2024-05-03T20:26:00.094 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.959+0000 7f3a6f0f9700 1 --2- 172.21.15.181:0/651123354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a68108f10 0x7f3a68109350 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:00.095 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.959+0000 7f3a6f0f9700 1 -- 172.21.15.181:0/651123354 >> 172.21.15.181:0/651123354 conn(0x7f3a68104520 msgr2=0x7f3a68106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:00.095 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.959+0000 7f3a6f0f9700 1 -- 172.21.15.181:0/651123354 shutdown_connections 2024-05-03T20:26:00.095 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.959+0000 7f3a6f0f9700 1 -- 172.21.15.181:0/651123354 wait complete. 2024-05-03T20:26:00.095 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.960+0000 7f3a6f0f9700 1 Processor -- start 2024-05-03T20:26:00.095 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.960+0000 7f3a6f0f9700 1 -- start start 2024-05-03T20:26:00.095 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.960+0000 7f3a6f0f9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a68108f10 0x7f3a6819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:00.095 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.960+0000 7f3a6f0f9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6810af10 con 0x7f3a68108f10 2024-05-03T20:26:00.095 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.960+0000 7f3a6ce95700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a68108f10 0x7f3a6819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:00.095 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.960+0000 7f3a6ce95700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a68108f10 0x7f3a6819cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60508/0 (socket says 172.21.15.181:60508) 2024-05-03T20:26:00.095 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.960+0000 7f3a6ce95700 1 -- 172.21.15.181:0/3715620813 learned_addr learned my addr 172.21.15.181:0/3715620813 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:00.095 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.961+0000 7f3a6ce95700 1 -- 172.21.15.181:0/3715620813 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6819d2e0 con 0x7f3a68108f10 2024-05-03T20:26:00.095 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.961+0000 7f3a6ce95700 1 --2- 172.21.15.181:0/3715620813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a68108f10 0x7f3a6819cda0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c000c00 tx=0x7f3a5c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:00.095 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.961+0000 7f3a65ffb700 1 -- 172.21.15.181:0/3715620813 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a5c003b20 con 0x7f3a68108f10 2024-05-03T20:26:00.095 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.961+0000 7f3a65ffb700 1 -- 172.21.15.181:0/3715620813 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3a5c01a8c0 con 0x7f3a68108f10 2024-05-03T20:26:00.096 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.961+0000 7f3a65ffb700 1 -- 172.21.15.181:0/3715620813 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a5c024940 con 0x7f3a68108f10 2024-05-03T20:26:00.096 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.961+0000 7f3a6f0f9700 1 -- 172.21.15.181:0/3715620813 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a6819d570 con 0x7f3a68108f10 2024-05-03T20:26:00.096 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.961+0000 7f3a6f0f9700 1 -- 172.21.15.181:0/3715620813 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a681a0220 con 0x7f3a68108f10 2024-05-03T20:26:00.096 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.962+0000 7f3a65ffb700 1 -- 172.21.15.181:0/3715620813 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f3a5c022020 con 0x7f3a68108f10 2024-05-03T20:26:00.096 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.962+0000 7f3a6f0f9700 1 -- 172.21.15.181:0/3715620813 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a68064b00 con 0x7f3a68108f10 2024-05-03T20:26:00.096 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.962+0000 7f3a65ffb700 1 --2- 172.21.15.181:0/3715620813 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7f3a5803e4e0 0x7f3a580409a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:00.096 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.962+0000 7f3a65ffb700 1 -- 172.21.15.181:0/3715620813 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3a5c0533f0 con 0x7f3a68108f10 2024-05-03T20:26:00.096 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.962+0000 7f3a67fff700 1 --2- 172.21.15.181:0/3715620813 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7f3a5803e4e0 0x7f3a580409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:00.096 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.963+0000 7f3a67fff700 1 --2- 172.21.15.181:0/3715620813 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7f3a5803e4e0 0x7f3a580409a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3a54006fd0 tx=0x7f3a54006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:00.096 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:58.967+0000 7f3a65ffb700 1 -- 172.21.15.181:0/3715620813 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3a5c0042a0 con 0x7f3a68108f10 2024-05-03T20:26:00.096 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:59.228+0000 7f3a6f0f9700 1 -- 172.21.15.181:0/3715620813 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f3a681a0440 con 0x7f3a68108f10 2024-05-03T20:26:00.096 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:59.471+0000 7f3a65ffb700 1 -- 172.21.15.181:0/3715620813 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f3a681a0440 con 0x7f3a68108f10 2024-05-03T20:26:00.096 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:59.472+0000 7f3a65ffb700 1 -- 172.21.15.181:0/3715620813 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f3a5c04d090 con 0x7f3a68108f10 2024-05-03T20:26:00.096 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:59.474+0000 7f3a6f0f9700 1 -- 172.21.15.181:0/3715620813 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7f3a5803e4e0 msgr2=0x7f3a580409a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:00.096 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:59.474+0000 7f3a6f0f9700 1 --2- 172.21.15.181:0/3715620813 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7f3a5803e4e0 0x7f3a580409a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3a54006fd0 tx=0x7f3a54006e40 comp rx=0 tx=0).stop 2024-05-03T20:26:00.097 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:59.474+0000 7f3a6f0f9700 1 -- 172.21.15.181:0/3715620813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a68108f10 msgr2=0x7f3a6819cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:00.097 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:59.474+0000 7f3a6f0f9700 1 --2- 172.21.15.181:0/3715620813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a68108f10 0x7f3a6819cda0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c000c00 tx=0x7f3a5c011b70 comp rx=0 tx=0).stop 2024-05-03T20:26:00.097 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:59.474+0000 7f3a6f0f9700 1 -- 172.21.15.181:0/3715620813 shutdown_connections 2024-05-03T20:26:00.097 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:59.474+0000 7f3a6f0f9700 1 --2- 172.21.15.181:0/3715620813 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7f3a5803e4e0 0x7f3a580409a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:00.097 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:59.474+0000 7f3a6f0f9700 1 --2- 172.21.15.181:0/3715620813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a68108f10 0x7f3a6819cda0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:00.097 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:59.474+0000 7f3a6f0f9700 1 -- 172.21.15.181:0/3715620813 >> 172.21.15.181:0/3715620813 conn(0x7f3a68104520 msgr2=0x7f3a68104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:00.097 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:59.474+0000 7f3a6f0f9700 1 -- 172.21.15.181:0/3715620813 shutdown_connections 2024-05-03T20:26:00.097 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:25:59.474+0000 7f3a6f0f9700 1 -- 172.21.15.181:0/3715620813 wait complete. 2024-05-03T20:26:00.336 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:00 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:00.244+0000 7f0142b8c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T20:26:00.336 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:00 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:00.330+0000 7f0142b8c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T20:26:00.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:00 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3715620813' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-03T20:26:00.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:00 smithi181 ceph-mon[29801]: mgrmap e5: x(active, since 6s) 2024-05-03T20:26:01.510 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout { 2024-05-03T20:26:01.510 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-03T20:26:01.510 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-03T20:26:01.510 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-03T20:26:01.510 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-03T20:26:01.510 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout } 2024-05-03T20:26:01.510 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.565+0000 7efe6f37f700 1 Processor -- start 2024-05-03T20:26:01.511 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.565+0000 7efe6f37f700 1 -- start start 2024-05-03T20:26:01.511 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.566+0000 7efe6f37f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe681099e0 0x7efe6810bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:01.511 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.566+0000 7efe6f37f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efe6807ce90 con 0x7efe681099e0 2024-05-03T20:26:01.511 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.566+0000 7efe6d11b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe681099e0 0x7efe6810bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:01.511 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.566+0000 7efe6d11b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe681099e0 0x7efe6810bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60532/0 (socket says 172.21.15.181:60532) 2024-05-03T20:26:01.511 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.566+0000 7efe6d11b700 1 -- 172.21.15.181:0/753757481 learned_addr learned my addr 172.21.15.181:0/753757481 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:01.511 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.566+0000 7efe6d11b700 1 -- 172.21.15.181:0/753757481 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6810c310 con 0x7efe681099e0 2024-05-03T20:26:01.511 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.566+0000 7efe6d11b700 1 --2- 172.21.15.181:0/753757481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe681099e0 0x7efe6810bdd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efe640098a0 tx=0x7efe64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801f821d28b2c259 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:01.511 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.566+0000 7efe5ffff700 1 -- 172.21.15.181:0/753757481 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe64004160 con 0x7efe681099e0 2024-05-03T20:26:01.511 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.566+0000 7efe5ffff700 1 -- 172.21.15.181:0/753757481 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efe64004300 con 0x7efe681099e0 2024-05-03T20:26:01.511 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.566+0000 7efe5ffff700 1 -- 172.21.15.181:0/753757481 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe64011af0 con 0x7efe681099e0 2024-05-03T20:26:01.511 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.567+0000 7efe6f37f700 1 -- 172.21.15.181:0/753757481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe681099e0 msgr2=0x7efe6810bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:01.511 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.567+0000 7efe6f37f700 1 --2- 172.21.15.181:0/753757481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe681099e0 0x7efe6810bdd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efe640098a0 tx=0x7efe64009bb0 comp rx=0 tx=0).stop 2024-05-03T20:26:01.511 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.567+0000 7efe6f37f700 1 -- 172.21.15.181:0/753757481 shutdown_connections 2024-05-03T20:26:01.511 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.567+0000 7efe6f37f700 1 --2- 172.21.15.181:0/753757481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe681099e0 0x7efe6810bdd0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:01.512 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.567+0000 7efe6f37f700 1 -- 172.21.15.181:0/753757481 >> 172.21.15.181:0/753757481 conn(0x7efe68103450 msgr2=0x7efe68105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:01.512 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.567+0000 7efe6f37f700 1 -- 172.21.15.181:0/753757481 shutdown_connections 2024-05-03T20:26:01.512 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.567+0000 7efe6f37f700 1 -- 172.21.15.181:0/753757481 wait complete. 2024-05-03T20:26:01.512 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.568+0000 7efe6f37f700 1 Processor -- start 2024-05-03T20:26:01.512 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.568+0000 7efe6f37f700 1 -- start start 2024-05-03T20:26:01.512 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.568+0000 7efe6f37f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe681099e0 0x7efe68109260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:01.512 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.568+0000 7efe6f37f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efe6810d270 con 0x7efe681099e0 2024-05-03T20:26:01.512 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.568+0000 7efe6d11b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe681099e0 0x7efe68109260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:01.512 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.568+0000 7efe6d11b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe681099e0 0x7efe68109260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60542/0 (socket says 172.21.15.181:60542) 2024-05-03T20:26:01.512 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.568+0000 7efe6d11b700 1 -- 172.21.15.181:0/3671009750 learned_addr learned my addr 172.21.15.181:0/3671009750 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:01.512 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.568+0000 7efe6d11b700 1 -- 172.21.15.181:0/3671009750 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe681097a0 con 0x7efe681099e0 2024-05-03T20:26:01.512 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.568+0000 7efe6d11b700 1 --2- 172.21.15.181:0/3671009750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe681099e0 0x7efe68109260 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efe64011ac0 tx=0x7efe64011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:01.512 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.569+0000 7efe5e7fc700 1 -- 172.21.15.181:0/3671009750 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe64003970 con 0x7efe681099e0 2024-05-03T20:26:01.512 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.569+0000 7efe6f37f700 1 -- 172.21.15.181:0/3671009750 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe68107990 con 0x7efe681099e0 2024-05-03T20:26:01.512 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.569+0000 7efe5e7fc700 1 -- 172.21.15.181:0/3671009750 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efe64003b10 con 0x7efe681099e0 2024-05-03T20:26:01.513 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.569+0000 7efe5e7fc700 1 -- 172.21.15.181:0/3671009750 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe64024610 con 0x7efe681099e0 2024-05-03T20:26:01.513 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.569+0000 7efe6f37f700 1 -- 172.21.15.181:0/3671009750 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe68107ed0 con 0x7efe681099e0 2024-05-03T20:26:01.513 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.569+0000 7efe5e7fc700 1 -- 172.21.15.181:0/3671009750 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7efe64024890 con 0x7efe681099e0 2024-05-03T20:26:01.513 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.570+0000 7efe5e7fc700 1 --2- 172.21.15.181:0/3671009750 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7efe5403e530 0x7efe540409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:01.513 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.570+0000 7efe6f37f700 1 -- 172.21.15.181:0/3671009750 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe68064b00 con 0x7efe681099e0 2024-05-03T20:26:01.513 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.570+0000 7efe5e7fc700 1 -- 172.21.15.181:0/3671009750 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efe64053890 con 0x7efe681099e0 2024-05-03T20:26:01.513 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.570+0000 7efe6c91a700 1 -- 172.21.15.181:0/3671009750 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7efe5403e530 msgr2=0x7efe540409f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6800/1802984663 2024-05-03T20:26:01.513 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.570+0000 7efe6c91a700 1 --2- 172.21.15.181:0/3671009750 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7efe5403e530 0x7efe540409f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T20:26:01.513 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.575+0000 7efe5e7fc700 1 -- 172.21.15.181:0/3671009750 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7efe6402c350 con 0x7efe681099e0 2024-05-03T20:26:01.513 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.771+0000 7efe6c91a700 1 -- 172.21.15.181:0/3671009750 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7efe5403e530 msgr2=0x7efe540409f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6800/1802984663 2024-05-03T20:26:01.513 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.771+0000 7efe6c91a700 1 --2- 172.21.15.181:0/3671009750 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7efe5403e530 0x7efe540409f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-03T20:26:01.513 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.877+0000 7efe6f37f700 1 -- 172.21.15.181:0/3671009750 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7efe68108680 con 0x7efe681099e0 2024-05-03T20:26:01.513 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.878+0000 7efe5e7fc700 1 -- 172.21.15.181:0/3671009750 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7efe640172b0 con 0x7efe681099e0 2024-05-03T20:26:01.513 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.880+0000 7efe6f37f700 1 -- 172.21.15.181:0/3671009750 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7efe5403e530 msgr2=0x7efe540409f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T20:26:01.513 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.880+0000 7efe6f37f700 1 --2- 172.21.15.181:0/3671009750 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7efe5403e530 0x7efe540409f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:01.514 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.880+0000 7efe6f37f700 1 -- 172.21.15.181:0/3671009750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe681099e0 msgr2=0x7efe68109260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:01.514 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.880+0000 7efe6f37f700 1 --2- 172.21.15.181:0/3671009750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe681099e0 0x7efe68109260 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efe64011ac0 tx=0x7efe64011ba0 comp rx=0 tx=0).stop 2024-05-03T20:26:01.514 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.880+0000 7efe6f37f700 1 -- 172.21.15.181:0/3671009750 shutdown_connections 2024-05-03T20:26:01.514 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.880+0000 7efe6f37f700 1 --2- 172.21.15.181:0/3671009750 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7efe5403e530 0x7efe540409f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:01.514 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.880+0000 7efe6f37f700 1 --2- 172.21.15.181:0/3671009750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe681099e0 0x7efe68109260 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:01.514 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.880+0000 7efe6f37f700 1 -- 172.21.15.181:0/3671009750 >> 172.21.15.181:0/3671009750 conn(0x7efe68103450 msgr2=0x7efe68103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:01.514 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.880+0000 7efe6f37f700 1 -- 172.21.15.181:0/3671009750 shutdown_connections 2024-05-03T20:26:01.514 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:00.880+0000 7efe6f37f700 1 -- 172.21.15.181:0/3671009750 wait complete. 2024-05-03T20:26:01.514 INFO:teuthology.orchestra.run.smithi181.stdout:Waiting for the mgr to restart... 2024-05-03T20:26:01.514 INFO:teuthology.orchestra.run.smithi181.stdout:Waiting for mgr epoch 5... 2024-05-03T20:26:01.727 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:01 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3671009750' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-03T20:26:01.727 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:01 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:01.555+0000 7f0142b8c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T20:26:01.727 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:01 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:01.727+0000 7f0142b8c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T20:26:02.126 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:01 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:01.903+0000 7f0142b8c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T20:26:02.463 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:02 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:02.195+0000 7f0142b8c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T20:26:02.464 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:02 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:02.287+0000 7f0142b8c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T20:26:02.963 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:02 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:02.490+0000 7f0142b8c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T20:26:02.963 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:02 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:02.680+0000 7f0142b8c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T20:26:03.713 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:03 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:03.281+0000 7f0142b8c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T20:26:03.713 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:03 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:03.501+0000 7f0142b8c200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T20:26:04.213 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:03 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:03.922+0000 7f0142b8c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T20:26:04.213 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:04 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:04.018+0000 7f0142b8c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T20:26:04.713 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:04 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:04.389+0000 7f0142b8c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T20:26:05.080 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:04 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:04.742+0000 7f0142b8c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T20:26:05.080 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:04 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:04.837+0000 7f0142b8c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T20:26:05.463 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:05 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:05.080+0000 7f0142b8c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T20:26:05.463 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:05 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:05.215+0000 7f0142b8c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T20:26:05.963 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:05 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:05.525+0000 7f0142b8c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T20:26:05.963 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:05 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:05.641+0000 7f0142b8c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T20:26:05.963 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:05 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:05.733+0000 7f0142b8c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T20:26:06.612 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:06 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:06.338+0000 7f0142b8c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T20:26:06.613 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:06 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:06.559+0000 7f0142b8c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T20:26:06.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:06 smithi181 ceph-mon[29801]: Active manager daemon x restarted 2024-05-03T20:26:06.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:06 smithi181 ceph-mon[29801]: Activating manager daemon x 2024-05-03T20:26:06.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:06 smithi181 ceph-mon[29801]: osdmap e2: 0 total, 0 up, 0 in 2024-05-03T20:26:06.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:06 smithi181 ceph-mon[29801]: mgrmap e6: x(active, starting, since 0.00389018s) 2024-05-03T20:26:06.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:06 smithi181 ceph-mon[29801]: from='mgr.14120 172.21.15.181:0/2717494325' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T20:26:06.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:06 smithi181 ceph-mon[29801]: from='mgr.14120 172.21.15.181:0/2717494325' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-03T20:26:06.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:06 smithi181 ceph-mon[29801]: from='mgr.14120 172.21.15.181:0/2717494325' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-03T20:26:06.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:06 smithi181 ceph-mon[29801]: from='mgr.14120 172.21.15.181:0/2717494325' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-03T20:26:06.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:06 smithi181 ceph-mon[29801]: from='mgr.14120 172.21.15.181:0/2717494325' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-03T20:26:06.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:06 smithi181 ceph-mon[29801]: Manager daemon x is now available 2024-05-03T20:26:06.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:06 smithi181 ceph-mon[29801]: from='mgr.14120 172.21.15.181:0/2717494325' entity='mgr.x' 2024-05-03T20:26:06.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:06 smithi181 ceph-mon[29801]: from='mgr.14120 172.21.15.181:0/2717494325' entity='mgr.x' 2024-05-03T20:26:06.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:06 smithi181 ceph-mon[29801]: from='mgr.14120 172.21.15.181:0/2717494325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:26:06.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:06 smithi181 ceph-mon[29801]: from='mgr.14120 172.21.15.181:0/2717494325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:26:06.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:06 smithi181 ceph-mon[29801]: from='mgr.14120 172.21.15.181:0/2717494325' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-03T20:26:07.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:07 smithi181 ceph-mon[29801]: Found migration_current of "None". Setting to last migration. 2024-05-03T20:26:07.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:07 smithi181 ceph-mon[29801]: from='mgr.14120 172.21.15.181:0/2717494325' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-03T20:26:07.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:07 smithi181 ceph-mon[29801]: from='mgr.14120 172.21.15.181:0/2717494325' entity='mgr.x' 2024-05-03T20:26:07.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:07 smithi181 ceph-mon[29801]: from='mgr.14120 172.21.15.181:0/2717494325' entity='mgr.x' 2024-05-03T20:26:07.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:07 smithi181 ceph-mon[29801]: mgrmap e7: x(active, since 1.00672s) 2024-05-03T20:26:08.192 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout { 2024-05-03T20:26:08.192 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-03T20:26:08.192 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-03T20:26:08.192 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout } 2024-05-03T20:26:08.193 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.920+0000 7f663fc09700 1 Processor -- start 2024-05-03T20:26:08.193 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.921+0000 7f663fc09700 1 -- start start 2024-05-03T20:26:08.193 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.921+0000 7f663fc09700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6638107e70 0x7f6638108290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:08.193 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.921+0000 7f663fc09700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f66381087d0 con 0x7f6638107e70 2024-05-03T20:26:08.193 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.921+0000 7f663d9a5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6638107e70 0x7f6638108290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:08.193 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.921+0000 7f663d9a5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6638107e70 0x7f6638108290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60554/0 (socket says 172.21.15.181:60554) 2024-05-03T20:26:08.193 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.921+0000 7f663d9a5700 1 -- 172.21.15.181:0/508896985 learned_addr learned my addr 172.21.15.181:0/508896985 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:08.193 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.921+0000 7f663d9a5700 1 -- 172.21.15.181:0/508896985 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6638108fd0 con 0x7f6638107e70 2024-05-03T20:26:08.193 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.921+0000 7f663d9a5700 1 --2- 172.21.15.181:0/508896985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6638107e70 0x7f6638108290 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f66340098a0 tx=0x7f6634009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f281aa12738c935 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:08.193 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.922+0000 7f663c9a3700 1 -- 172.21.15.181:0/508896985 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6634004160 con 0x7f6638107e70 2024-05-03T20:26:08.193 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.922+0000 7f663c9a3700 1 -- 172.21.15.181:0/508896985 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6634004300 con 0x7f6638107e70 2024-05-03T20:26:08.193 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.922+0000 7f663c9a3700 1 -- 172.21.15.181:0/508896985 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6634011af0 con 0x7f6638107e70 2024-05-03T20:26:08.193 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.922+0000 7f663fc09700 1 -- 172.21.15.181:0/508896985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6638107e70 msgr2=0x7f6638108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:08.193 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.922+0000 7f663fc09700 1 --2- 172.21.15.181:0/508896985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6638107e70 0x7f6638108290 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f66340098a0 tx=0x7f6634009bb0 comp rx=0 tx=0).stop 2024-05-03T20:26:08.194 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.922+0000 7f663fc09700 1 -- 172.21.15.181:0/508896985 shutdown_connections 2024-05-03T20:26:08.194 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.922+0000 7f663fc09700 1 --2- 172.21.15.181:0/508896985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6638107e70 0x7f6638108290 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:08.194 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.922+0000 7f663fc09700 1 -- 172.21.15.181:0/508896985 >> 172.21.15.181:0/508896985 conn(0x7f6638103450 msgr2=0x7f6638105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:08.194 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.922+0000 7f663fc09700 1 -- 172.21.15.181:0/508896985 shutdown_connections 2024-05-03T20:26:08.194 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.922+0000 7f663fc09700 1 -- 172.21.15.181:0/508896985 wait complete. 2024-05-03T20:26:08.194 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.923+0000 7f663fc09700 1 Processor -- start 2024-05-03T20:26:08.194 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.923+0000 7f663fc09700 1 -- start start 2024-05-03T20:26:08.194 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.923+0000 7f663fc09700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6638107e70 0x7f663819bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:08.194 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.923+0000 7f663fc09700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f663810c290 con 0x7f6638107e70 2024-05-03T20:26:08.194 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.923+0000 7f663d9a5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6638107e70 0x7f663819bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:08.194 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.923+0000 7f663d9a5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6638107e70 0x7f663819bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60556/0 (socket says 172.21.15.181:60556) 2024-05-03T20:26:08.194 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.923+0000 7f663d9a5700 1 -- 172.21.15.181:0/1726137374 learned_addr learned my addr 172.21.15.181:0/1726137374 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:08.194 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.924+0000 7f663d9a5700 1 -- 172.21.15.181:0/1726137374 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663819c220 con 0x7f6638107e70 2024-05-03T20:26:08.195 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.924+0000 7f663d9a5700 1 --2- 172.21.15.181:0/1726137374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6638107e70 0x7f663819bce0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6634011670 tx=0x7f6634004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:08.195 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.924+0000 7f662effd700 1 -- 172.21.15.181:0/1726137374 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66340036a0 con 0x7f6638107e70 2024-05-03T20:26:08.195 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.924+0000 7f662effd700 1 -- 172.21.15.181:0/1726137374 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6634003cd0 con 0x7f6638107e70 2024-05-03T20:26:08.195 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.924+0000 7f663fc09700 1 -- 172.21.15.181:0/1726137374 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f663819c4b0 con 0x7f6638107e70 2024-05-03T20:26:08.195 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.924+0000 7f663fc09700 1 -- 172.21.15.181:0/1726137374 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f663819f190 con 0x7f6638107e70 2024-05-03T20:26:08.195 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.924+0000 7f662effd700 1 -- 172.21.15.181:0/1726137374 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6634024e40 con 0x7f6638107e70 2024-05-03T20:26:08.195 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.925+0000 7f662effd700 1 -- 172.21.15.181:0/1726137374 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f663401a5f0 con 0x7f6638107e70 2024-05-03T20:26:08.195 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.925+0000 7f662effd700 1 --2- 172.21.15.181:0/1726137374 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7f662403e4e0 0x7f66240409a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:08.195 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.925+0000 7f663fc09700 1 -- 172.21.15.181:0/1726137374 --> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6638064b00 con 0x7f662403e4e0 2024-05-03T20:26:08.195 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.925+0000 7f662effd700 1 -- 172.21.15.181:0/1726137374 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f66340537c0 con 0x7f6638107e70 2024-05-03T20:26:08.195 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.925+0000 7f663d1a4700 1 -- 172.21.15.181:0/1726137374 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7f662403e4e0 msgr2=0x7f66240409a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6800/1802984663 2024-05-03T20:26:08.195 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:01.925+0000 7f663d1a4700 1 --2- 172.21.15.181:0/1726137374 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7f662403e4e0 0x7f66240409a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T20:26:08.195 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:02.126+0000 7f663d1a4700 1 -- 172.21.15.181:0/1726137374 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7f662403e4e0 msgr2=0x7f66240409a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6800/1802984663 2024-05-03T20:26:08.195 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:02.126+0000 7f663d1a4700 1 --2- 172.21.15.181:0/1726137374 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7f662403e4e0 0x7f66240409a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-03T20:26:08.196 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:02.527+0000 7f663d1a4700 1 -- 172.21.15.181:0/1726137374 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7f662403e4e0 msgr2=0x7f66240409a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6800/1802984663 2024-05-03T20:26:08.196 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:02.527+0000 7f663d1a4700 1 --2- 172.21.15.181:0/1726137374 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7f662403e4e0 0x7f66240409a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-03T20:26:08.196 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:03.328+0000 7f663d1a4700 1 -- 172.21.15.181:0/1726137374 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7f662403e4e0 msgr2=0x7f66240409a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6800/1802984663 2024-05-03T20:26:08.196 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:03.328+0000 7f663d1a4700 1 --2- 172.21.15.181:0/1726137374 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7f662403e4e0 0x7f66240409a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-03T20:26:08.196 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:04.929+0000 7f663d1a4700 1 -- 172.21.15.181:0/1726137374 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7f662403e4e0 msgr2=0x7f66240409a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6800/1802984663 2024-05-03T20:26:08.196 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:04.929+0000 7f663d1a4700 1 --2- 172.21.15.181:0/1726137374 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7f662403e4e0 0x7f66240409a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-03T20:26:08.196 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:06.563+0000 7f662effd700 1 -- 172.21.15.181:0/1726137374 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7f663401a8e0 con 0x7f6638107e70 2024-05-03T20:26:08.196 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:06.563+0000 7f662effd700 1 -- 172.21.15.181:0/1726137374 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7f662403e4e0 msgr2=0x7f66240409a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T20:26:08.196 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:06.563+0000 7f662effd700 1 --2- 172.21.15.181:0/1726137374 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7f662403e4e0 0x7f66240409a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:08.196 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:07.564+0000 7f662effd700 1 -- 172.21.15.181:0/1726137374 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f6634055590 con 0x7f6638107e70 2024-05-03T20:26:08.196 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:07.564+0000 7f662effd700 1 --2- 172.21.15.181:0/1726137374 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f6624041be0 0x7f6624043fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:08.196 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:07.565+0000 7f662effd700 1 -- 172.21.15.181:0/1726137374 --> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f663401be50 con 0x7f6624041be0 2024-05-03T20:26:08.196 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:07.565+0000 7f663d1a4700 1 --2- 172.21.15.181:0/1726137374 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f6624041be0 0x7f6624043fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:08.196 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:07.566+0000 7f663d1a4700 1 --2- 172.21.15.181:0/1726137374 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f6624041be0 0x7f6624043fd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6628003a10 tx=0x7f6628009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:08.197 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:07.566+0000 7f662effd700 1 -- 172.21.15.181:0/1726137374 <== mgr.14120 v2:172.21.15.181:6800/2389231557 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f663401be50 con 0x7f6624041be0 2024-05-03T20:26:08.197 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:07.575+0000 7f663fc09700 1 -- 172.21.15.181:0/1726137374 --> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f6638104590 con 0x7f6624041be0 2024-05-03T20:26:08.197 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:07.575+0000 7f662effd700 1 -- 172.21.15.181:0/1726137374 <== mgr.14120 v2:172.21.15.181:6800/2389231557 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f6638104590 con 0x7f6624041be0 2024-05-03T20:26:08.197 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:07.575+0000 7f663fc09700 1 -- 172.21.15.181:0/1726137374 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f6624041be0 msgr2=0x7f6624043fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:08.197 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:07.575+0000 7f663fc09700 1 --2- 172.21.15.181:0/1726137374 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f6624041be0 0x7f6624043fd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6628003a10 tx=0x7f6628009290 comp rx=0 tx=0).stop 2024-05-03T20:26:08.197 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:07.575+0000 7f663fc09700 1 -- 172.21.15.181:0/1726137374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6638107e70 msgr2=0x7f663819bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:08.197 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:07.575+0000 7f663fc09700 1 --2- 172.21.15.181:0/1726137374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6638107e70 0x7f663819bce0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6634011670 tx=0x7f6634004250 comp rx=0 tx=0).stop 2024-05-03T20:26:08.197 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:07.575+0000 7f663fc09700 1 -- 172.21.15.181:0/1726137374 shutdown_connections 2024-05-03T20:26:08.197 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:07.575+0000 7f663fc09700 1 --2- 172.21.15.181:0/1726137374 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f6624041be0 0x7f6624043fd0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:08.197 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:07.575+0000 7f663fc09700 1 --2- 172.21.15.181:0/1726137374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6638107e70 0x7f663819bce0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:08.197 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:07.575+0000 7f663fc09700 1 --2- 172.21.15.181:0/1726137374 >> [v2:172.21.15.181:6800/1802984663,v1:172.21.15.181:6801/1802984663] conn(0x7f662403e4e0 0x7f66240409a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:08.197 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:07.575+0000 7f663fc09700 1 -- 172.21.15.181:0/1726137374 >> 172.21.15.181:0/1726137374 conn(0x7f6638103450 msgr2=0x7f6638103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:08.197 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:07.575+0000 7f663fc09700 1 -- 172.21.15.181:0/1726137374 shutdown_connections 2024-05-03T20:26:08.197 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:07.575+0000 7f663fc09700 1 -- 172.21.15.181:0/1726137374 wait complete. 2024-05-03T20:26:08.197 INFO:teuthology.orchestra.run.smithi181.stdout:mgr epoch 5 is available 2024-05-03T20:26:08.198 INFO:teuthology.orchestra.run.smithi181.stdout:Setting orchestrator backend to cephadm... 2024-05-03T20:26:08.846 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:08 smithi181 ceph-mon[29801]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-03T20:26:08.847 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:08 smithi181 ceph-mon[29801]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-03T20:26:08.847 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:08 smithi181 ceph-mon[29801]: [03/May/2024:20:26:07] ENGINE Bus STARTING 2024-05-03T20:26:08.847 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:08 smithi181 ceph-mon[29801]: [03/May/2024:20:26:07] ENGINE Serving on http://172.21.15.181:8765 2024-05-03T20:26:08.847 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:08 smithi181 ceph-mon[29801]: [03/May/2024:20:26:08] ENGINE Serving on https://172.21.15.181:7150 2024-05-03T20:26:08.847 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:08 smithi181 ceph-mon[29801]: [03/May/2024:20:26:08] ENGINE Bus STARTED 2024-05-03T20:26:08.847 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:08 smithi181 ceph-mon[29801]: from='mgr.14120 172.21.15.181:0/2717494325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:26:09.422 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.620+0000 7f3e6778a700 1 Processor -- start 2024-05-03T20:26:09.422 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.620+0000 7f3e6778a700 1 -- start start 2024-05-03T20:26:09.422 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.620+0000 7f3e6778a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e60107250 0x7f3e60107670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:09.422 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.620+0000 7f3e6778a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3e60107c40 con 0x7f3e60107250 2024-05-03T20:26:09.422 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.620+0000 7f3e65526700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e60107250 0x7f3e60107670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:09.422 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.620+0000 7f3e65526700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e60107250 0x7f3e60107670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57494/0 (socket says 172.21.15.181:57494) 2024-05-03T20:26:09.422 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.620+0000 7f3e65526700 1 -- 172.21.15.181:0/439771880 learned_addr learned my addr 172.21.15.181:0/439771880 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:09.422 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.620+0000 7f3e65526700 1 -- 172.21.15.181:0/439771880 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6010a1e0 con 0x7f3e60107250 2024-05-03T20:26:09.423 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.621+0000 7f3e65526700 1 --2- 172.21.15.181:0/439771880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e60107250 0x7f3e60107670 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c0098a0 tx=0x7f3e5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30effc5df383c2ab server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:09.423 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.621+0000 7f3e57fff700 1 -- 172.21.15.181:0/439771880 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e5c004160 con 0x7f3e60107250 2024-05-03T20:26:09.423 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.621+0000 7f3e57fff700 1 -- 172.21.15.181:0/439771880 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3e5c004300 con 0x7f3e60107250 2024-05-03T20:26:09.423 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.621+0000 7f3e57fff700 1 -- 172.21.15.181:0/439771880 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e5c011af0 con 0x7f3e60107250 2024-05-03T20:26:09.423 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.621+0000 7f3e6778a700 1 -- 172.21.15.181:0/439771880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e60107250 msgr2=0x7f3e60107670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:09.423 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.621+0000 7f3e6778a700 1 --2- 172.21.15.181:0/439771880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e60107250 0x7f3e60107670 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c0098a0 tx=0x7f3e5c009bb0 comp rx=0 tx=0).stop 2024-05-03T20:26:09.423 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.621+0000 7f3e6778a700 1 -- 172.21.15.181:0/439771880 shutdown_connections 2024-05-03T20:26:09.423 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.621+0000 7f3e6778a700 1 --2- 172.21.15.181:0/439771880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e60107250 0x7f3e60107670 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:09.423 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.621+0000 7f3e6778a700 1 -- 172.21.15.181:0/439771880 >> 172.21.15.181:0/439771880 conn(0x7f3e60102c00 msgr2=0x7f3e60105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:09.423 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.622+0000 7f3e6778a700 1 -- 172.21.15.181:0/439771880 shutdown_connections 2024-05-03T20:26:09.423 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.622+0000 7f3e6778a700 1 -- 172.21.15.181:0/439771880 wait complete. 2024-05-03T20:26:09.423 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.622+0000 7f3e6778a700 1 Processor -- start 2024-05-03T20:26:09.423 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.622+0000 7f3e6778a700 1 -- start start 2024-05-03T20:26:09.423 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.622+0000 7f3e6778a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e60107250 0x7f3e6019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:09.423 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.622+0000 7f3e6778a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6010d410 con 0x7f3e60107250 2024-05-03T20:26:09.424 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.622+0000 7f3e65526700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e60107250 0x7f3e6019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:09.424 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.622+0000 7f3e65526700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e60107250 0x7f3e6019cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57508/0 (socket says 172.21.15.181:57508) 2024-05-03T20:26:09.424 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.622+0000 7f3e65526700 1 -- 172.21.15.181:0/3272039809 learned_addr learned my addr 172.21.15.181:0/3272039809 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:09.424 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.623+0000 7f3e65526700 1 -- 172.21.15.181:0/3272039809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6019d2f0 con 0x7f3e60107250 2024-05-03T20:26:09.424 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.623+0000 7f3e65526700 1 --2- 172.21.15.181:0/3272039809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e60107250 0x7f3e6019cdb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009fb0 tx=0x7f3e5c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:09.424 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.623+0000 7f3e567fc700 1 -- 172.21.15.181:0/3272039809 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e5c0036a0 con 0x7f3e60107250 2024-05-03T20:26:09.424 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.623+0000 7f3e567fc700 1 -- 172.21.15.181:0/3272039809 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3e5c01a430 con 0x7f3e60107250 2024-05-03T20:26:09.424 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.623+0000 7f3e6778a700 1 -- 172.21.15.181:0/3272039809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e6019d580 con 0x7f3e60107250 2024-05-03T20:26:09.424 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.623+0000 7f3e567fc700 1 -- 172.21.15.181:0/3272039809 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e5c019550 con 0x7f3e60107250 2024-05-03T20:26:09.424 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.623+0000 7f3e6778a700 1 -- 172.21.15.181:0/3272039809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e601a0270 con 0x7f3e60107250 2024-05-03T20:26:09.424 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.624+0000 7f3e567fc700 1 -- 172.21.15.181:0/3272039809 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f3e5c019800 con 0x7f3e60107250 2024-05-03T20:26:09.424 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.624+0000 7f3e6778a700 1 -- 172.21.15.181:0/3272039809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e60064b00 con 0x7f3e60107250 2024-05-03T20:26:09.424 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.624+0000 7f3e567fc700 1 --2- 172.21.15.181:0/3272039809 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f3e4c03e530 0x7f3e4c0409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:09.424 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.624+0000 7f3e567fc700 1 -- 172.21.15.181:0/3272039809 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3e5c053830 con 0x7f3e60107250 2024-05-03T20:26:09.424 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.624+0000 7f3e64d25700 1 --2- 172.21.15.181:0/3272039809 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f3e4c03e530 0x7f3e4c0409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:09.425 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.625+0000 7f3e64d25700 1 --2- 172.21.15.181:0/3272039809 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f3e4c03e530 0x7f3e4c0409f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3e50006fd0 tx=0x7f3e50006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:09.425 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.629+0000 7f3e567fc700 1 -- 172.21.15.181:0/3272039809 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3e5c017e90 con 0x7f3e60107250 2024-05-03T20:26:09.425 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.846+0000 7f3e6778a700 1 -- 172.21.15.181:0/3272039809 --> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f3e60103cd0 con 0x7f3e4c03e530 2024-05-03T20:26:09.425 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.851+0000 7f3e567fc700 1 -- 172.21.15.181:0/3272039809 <== mgr.14120 v2:172.21.15.181:6800/2389231557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3e60103cd0 con 0x7f3e4c03e530 2024-05-03T20:26:09.425 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.853+0000 7f3e6778a700 1 -- 172.21.15.181:0/3272039809 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f3e4c03e530 msgr2=0x7f3e4c0409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:09.425 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.853+0000 7f3e6778a700 1 --2- 172.21.15.181:0/3272039809 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f3e4c03e530 0x7f3e4c0409f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3e50006fd0 tx=0x7f3e50006e40 comp rx=0 tx=0).stop 2024-05-03T20:26:09.425 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.853+0000 7f3e6778a700 1 -- 172.21.15.181:0/3272039809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e60107250 msgr2=0x7f3e6019cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:09.425 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.853+0000 7f3e6778a700 1 --2- 172.21.15.181:0/3272039809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e60107250 0x7f3e6019cdb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009fb0 tx=0x7f3e5c011940 comp rx=0 tx=0).stop 2024-05-03T20:26:09.425 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.853+0000 7f3e6778a700 1 -- 172.21.15.181:0/3272039809 shutdown_connections 2024-05-03T20:26:09.425 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.853+0000 7f3e6778a700 1 --2- 172.21.15.181:0/3272039809 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f3e4c03e530 0x7f3e4c0409f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:09.425 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.853+0000 7f3e6778a700 1 --2- 172.21.15.181:0/3272039809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e60107250 0x7f3e6019cdb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:09.425 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.853+0000 7f3e6778a700 1 -- 172.21.15.181:0/3272039809 >> 172.21.15.181:0/3272039809 conn(0x7f3e60102c00 msgr2=0x7f3e601035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:09.425 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.853+0000 7f3e6778a700 1 -- 172.21.15.181:0/3272039809 shutdown_connections 2024-05-03T20:26:09.425 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:08.853+0000 7f3e6778a700 1 -- 172.21.15.181:0/3272039809 wait complete. 2024-05-03T20:26:09.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:09 smithi181 ceph-mon[29801]: mgrmap e8: x(active, since 2s) 2024-05-03T20:26:09.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:09 smithi181 ceph-mon[29801]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:26:09.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:09 smithi181 ceph-mon[29801]: from='mgr.14120 172.21.15.181:0/2717494325' entity='mgr.x' 2024-05-03T20:26:09.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:09 smithi181 ceph-mon[29801]: from='mgr.14120 172.21.15.181:0/2717494325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:26:10.636 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-03T20:26:10.636 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.840+0000 7f1245048700 1 Processor -- start 2024-05-03T20:26:10.637 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.840+0000 7f1245048700 1 -- start start 2024-05-03T20:26:10.637 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.840+0000 7f1245048700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1240108380 0x7f12401087a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:10.637 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.840+0000 7f1245048700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1240108d70 con 0x7f1240108380 2024-05-03T20:26:10.637 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.841+0000 7f123ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1240108380 0x7f12401087a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:10.637 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.841+0000 7f123ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1240108380 0x7f12401087a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57518/0 (socket says 172.21.15.181:57518) 2024-05-03T20:26:10.637 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.841+0000 7f123ed9d700 1 -- 172.21.15.181:0/425791297 learned_addr learned my addr 172.21.15.181:0/425791297 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:10.637 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.841+0000 7f123ed9d700 1 -- 172.21.15.181:0/425791297 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12401090a0 con 0x7f1240108380 2024-05-03T20:26:10.637 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.841+0000 7f123ed9d700 1 --2- 172.21.15.181:0/425791297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1240108380 0x7f12401087a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f12300098a0 tx=0x7f1230009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8969e14266e27d38 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:10.637 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.841+0000 7f123dd9b700 1 -- 172.21.15.181:0/425791297 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1230004160 con 0x7f1240108380 2024-05-03T20:26:10.637 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.841+0000 7f123dd9b700 1 -- 172.21.15.181:0/425791297 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1230004300 con 0x7f1240108380 2024-05-03T20:26:10.637 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.841+0000 7f123dd9b700 1 -- 172.21.15.181:0/425791297 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1230011af0 con 0x7f1240108380 2024-05-03T20:26:10.638 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.842+0000 7f1245048700 1 -- 172.21.15.181:0/425791297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1240108380 msgr2=0x7f12401087a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:10.638 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.842+0000 7f1245048700 1 --2- 172.21.15.181:0/425791297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1240108380 0x7f12401087a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f12300098a0 tx=0x7f1230009bb0 comp rx=0 tx=0).stop 2024-05-03T20:26:10.638 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.842+0000 7f1245048700 1 -- 172.21.15.181:0/425791297 shutdown_connections 2024-05-03T20:26:10.638 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.842+0000 7f1245048700 1 --2- 172.21.15.181:0/425791297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1240108380 0x7f12401087a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:10.638 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.842+0000 7f1245048700 1 -- 172.21.15.181:0/425791297 >> 172.21.15.181:0/425791297 conn(0x7f1240102be0 msgr2=0x7f1240105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:10.638 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.842+0000 7f1245048700 1 -- 172.21.15.181:0/425791297 shutdown_connections 2024-05-03T20:26:10.638 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.842+0000 7f1245048700 1 -- 172.21.15.181:0/425791297 wait complete. 2024-05-03T20:26:10.638 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.843+0000 7f1245048700 1 Processor -- start 2024-05-03T20:26:10.638 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.843+0000 7f1245048700 1 -- start start 2024-05-03T20:26:10.638 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.843+0000 7f1245048700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1240108380 0x7f124019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:10.638 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.843+0000 7f1245048700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f124010c2d0 con 0x7f1240108380 2024-05-03T20:26:10.638 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.843+0000 7f123ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1240108380 0x7f124019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:10.639 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.843+0000 7f123ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1240108380 0x7f124019cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57532/0 (socket says 172.21.15.181:57532) 2024-05-03T20:26:10.639 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.843+0000 7f123ed9d700 1 -- 172.21.15.181:0/1198495881 learned_addr learned my addr 172.21.15.181:0/1198495881 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:10.639 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.843+0000 7f123ed9d700 1 -- 172.21.15.181:0/1198495881 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124019d320 con 0x7f1240108380 2024-05-03T20:26:10.639 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.843+0000 7f123ed9d700 1 --2- 172.21.15.181:0/1198495881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1240108380 0x7f124019cde0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1230011ac0 tx=0x7f1230011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:10.639 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.844+0000 7f1227fff700 1 -- 172.21.15.181:0/1198495881 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1230003970 con 0x7f1240108380 2024-05-03T20:26:10.639 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.844+0000 7f1227fff700 1 -- 172.21.15.181:0/1198495881 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1230003b10 con 0x7f1240108380 2024-05-03T20:26:10.639 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.844+0000 7f1245048700 1 -- 172.21.15.181:0/1198495881 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f124019d5b0 con 0x7f1240108380 2024-05-03T20:26:10.639 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.844+0000 7f1227fff700 1 -- 172.21.15.181:0/1198495881 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1230024610 con 0x7f1240108380 2024-05-03T20:26:10.639 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.844+0000 7f1245048700 1 -- 172.21.15.181:0/1198495881 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12401a02a0 con 0x7f1240108380 2024-05-03T20:26:10.639 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.845+0000 7f1227fff700 1 -- 172.21.15.181:0/1198495881 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f1230024890 con 0x7f1240108380 2024-05-03T20:26:10.639 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.845+0000 7f1245048700 1 -- 172.21.15.181:0/1198495881 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f124010c450 con 0x7f1240108380 2024-05-03T20:26:10.639 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.845+0000 7f1227fff700 1 --2- 172.21.15.181:0/1198495881 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f122803e530 0x7f12280409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:10.640 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.845+0000 7f1227fff700 1 -- 172.21.15.181:0/1198495881 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1230053880 con 0x7f1240108380 2024-05-03T20:26:10.640 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.845+0000 7f123e59c700 1 --2- 172.21.15.181:0/1198495881 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f122803e530 0x7f12280409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:10.640 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.845+0000 7f123e59c700 1 --2- 172.21.15.181:0/1198495881 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f122803e530 0x7f12280409f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1234006fd0 tx=0x7f1234006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:10.640 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:09.850+0000 7f1227fff700 1 -- 172.21.15.181:0/1198495881 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f123002c350 con 0x7f1240108380 2024-05-03T20:26:10.640 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:10.050+0000 7f1245048700 1 -- 172.21.15.181:0/1198495881 --> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f124002e840 con 0x7f122803e530 2024-05-03T20:26:10.640 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:10.051+0000 7f1227fff700 1 -- 172.21.15.181:0/1198495881 <== mgr.14120 v2:172.21.15.181:6800/2389231557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f124002e840 con 0x7f122803e530 2024-05-03T20:26:10.640 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:10.053+0000 7f1245048700 1 -- 172.21.15.181:0/1198495881 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f122803e530 msgr2=0x7f12280409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:10.640 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:10.053+0000 7f1245048700 1 --2- 172.21.15.181:0/1198495881 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f122803e530 0x7f12280409f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1234006fd0 tx=0x7f1234006e40 comp rx=0 tx=0).stop 2024-05-03T20:26:10.640 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:10.053+0000 7f1245048700 1 -- 172.21.15.181:0/1198495881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1240108380 msgr2=0x7f124019cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:10.640 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:10.053+0000 7f1245048700 1 --2- 172.21.15.181:0/1198495881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1240108380 0x7f124019cde0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1230011ac0 tx=0x7f1230011ba0 comp rx=0 tx=0).stop 2024-05-03T20:26:10.640 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:10.053+0000 7f1245048700 1 -- 172.21.15.181:0/1198495881 shutdown_connections 2024-05-03T20:26:10.640 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:10.053+0000 7f1245048700 1 --2- 172.21.15.181:0/1198495881 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f122803e530 0x7f12280409f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:10.640 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:10.053+0000 7f1245048700 1 --2- 172.21.15.181:0/1198495881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1240108380 0x7f124019cde0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:10.641 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:10.053+0000 7f1245048700 1 -- 172.21.15.181:0/1198495881 >> 172.21.15.181:0/1198495881 conn(0x7f1240102be0 msgr2=0x7f12401039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:10.641 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:10.054+0000 7f1245048700 1 -- 172.21.15.181:0/1198495881 shutdown_connections 2024-05-03T20:26:10.641 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:10.054+0000 7f1245048700 1 -- 172.21.15.181:0/1198495881 wait complete. 2024-05-03T20:26:10.641 INFO:teuthology.orchestra.run.smithi181.stdout:Generating ssh key... 2024-05-03T20:26:10.947 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:10 smithi181 ceph-mon[29801]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:26:11.714 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:11 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: Generating public/private rsa key pair. 2024-05-03T20:26:11.714 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:11 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: Your identification has been saved in /tmp/tmpz445qpe1/key. 2024-05-03T20:26:11.714 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:11 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: Your public key has been saved in /tmp/tmpz445qpe1/key.pub. 2024-05-03T20:26:11.714 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:11 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: The key fingerprint is: 2024-05-03T20:26:11.714 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:11 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: SHA256:Cru2a0iQMLuuCUemSWypCK+2/6YwP295liUcRqmVc1c ceph-1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:26:11.714 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:11 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: The key's randomart image is: 2024-05-03T20:26:11.714 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:11 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: +---[RSA 3072]----+ 2024-05-03T20:26:11.714 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:11 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: | o .E | 2024-05-03T20:26:11.714 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:11 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: |o * . . | 2024-05-03T20:26:11.715 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:11 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: |.+ + o . | 2024-05-03T20:26:11.715 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:11 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: |= . . o | 2024-05-03T20:26:11.715 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:11 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: |.Bo . o S | 2024-05-03T20:26:11.715 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:11 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: |B*. o + . | 2024-05-03T20:26:11.715 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:11 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: |B+o.. o + | 2024-05-03T20:26:11.715 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:11 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: |.*=.o= + | 2024-05-03T20:26:11.715 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:11 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: |*.o*@=o | 2024-05-03T20:26:11.715 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:11 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: +----[SHA256]-----+ 2024-05-03T20:26:11.958 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.050+0000 7fe4e9f9f700 1 Processor -- start 2024-05-03T20:26:11.959 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.050+0000 7fe4e9f9f700 1 -- start start 2024-05-03T20:26:11.959 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.050+0000 7fe4e9f9f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4e4107cc0 0x7fe4e41080e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:11.959 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.050+0000 7fe4e9f9f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e41086b0 con 0x7fe4e4107cc0 2024-05-03T20:26:11.959 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.050+0000 7fe4e8f9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4e4107cc0 0x7fe4e41080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:11.959 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.050+0000 7fe4e8f9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4e4107cc0 0x7fe4e41080e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57534/0 (socket says 172.21.15.181:57534) 2024-05-03T20:26:11.959 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.050+0000 7fe4e8f9d700 1 -- 172.21.15.181:0/1410395629 learned_addr learned my addr 172.21.15.181:0/1410395629 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:11.959 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.050+0000 7fe4e8f9d700 1 -- 172.21.15.181:0/1410395629 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e4108ec0 con 0x7fe4e4107cc0 2024-05-03T20:26:11.959 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.051+0000 7fe4e8f9d700 1 --2- 172.21.15.181:0/1410395629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4e4107cc0 0x7fe4e41080e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80098a0 tx=0x7fe4d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e0d014dd794a5a server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:11.959 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.051+0000 7fe4e37fe700 1 -- 172.21.15.181:0/1410395629 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4d8004160 con 0x7fe4e4107cc0 2024-05-03T20:26:11.959 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.051+0000 7fe4e37fe700 1 -- 172.21.15.181:0/1410395629 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe4d8004300 con 0x7fe4e4107cc0 2024-05-03T20:26:11.960 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.051+0000 7fe4e37fe700 1 -- 172.21.15.181:0/1410395629 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4d8011af0 con 0x7fe4e4107cc0 2024-05-03T20:26:11.960 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.051+0000 7fe4e9f9f700 1 -- 172.21.15.181:0/1410395629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4e4107cc0 msgr2=0x7fe4e41080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:11.960 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.051+0000 7fe4e9f9f700 1 --2- 172.21.15.181:0/1410395629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4e4107cc0 0x7fe4e41080e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80098a0 tx=0x7fe4d8009bb0 comp rx=0 tx=0).stop 2024-05-03T20:26:11.960 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.051+0000 7fe4e9f9f700 1 -- 172.21.15.181:0/1410395629 shutdown_connections 2024-05-03T20:26:11.960 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.051+0000 7fe4e9f9f700 1 --2- 172.21.15.181:0/1410395629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4e4107cc0 0x7fe4e41080e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:11.960 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.051+0000 7fe4e9f9f700 1 -- 172.21.15.181:0/1410395629 >> 172.21.15.181:0/1410395629 conn(0x7fe4e4103280 msgr2=0x7fe4e41056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:11.960 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.051+0000 7fe4e9f9f700 1 -- 172.21.15.181:0/1410395629 shutdown_connections 2024-05-03T20:26:11.960 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.051+0000 7fe4e9f9f700 1 -- 172.21.15.181:0/1410395629 wait complete. 2024-05-03T20:26:11.960 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.052+0000 7fe4e9f9f700 1 Processor -- start 2024-05-03T20:26:11.960 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.052+0000 7fe4e9f9f700 1 -- start start 2024-05-03T20:26:11.960 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.052+0000 7fe4e9f9f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4e4107cc0 0x7fe4e419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:11.960 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.052+0000 7fe4e9f9f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e407bfb0 con 0x7fe4e4107cc0 2024-05-03T20:26:11.960 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.052+0000 7fe4e8f9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4e4107cc0 0x7fe4e419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:11.961 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.052+0000 7fe4e8f9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4e4107cc0 0x7fe4e419ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57544/0 (socket says 172.21.15.181:57544) 2024-05-03T20:26:11.961 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.053+0000 7fe4e8f9d700 1 -- 172.21.15.181:0/4069449169 learned_addr learned my addr 172.21.15.181:0/4069449169 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:11.961 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.053+0000 7fe4e8f9d700 1 -- 172.21.15.181:0/4069449169 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e419f390 con 0x7fe4e4107cc0 2024-05-03T20:26:11.961 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.053+0000 7fe4e8f9d700 1 --2- 172.21.15.181:0/4069449169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4e4107cc0 0x7fe4e419ee50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8009fb0 tx=0x7fe4d8011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:11.961 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.053+0000 7fe4e1ffb700 1 -- 172.21.15.181:0/4069449169 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4d80036a0 con 0x7fe4e4107cc0 2024-05-03T20:26:11.961 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.053+0000 7fe4e1ffb700 1 -- 172.21.15.181:0/4069449169 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe4d801a430 con 0x7fe4e4107cc0 2024-05-03T20:26:11.961 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.053+0000 7fe4e9f9f700 1 -- 172.21.15.181:0/4069449169 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e419f620 con 0x7fe4e4107cc0 2024-05-03T20:26:11.961 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.053+0000 7fe4e1ffb700 1 -- 172.21.15.181:0/4069449169 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4d8019550 con 0x7fe4e4107cc0 2024-05-03T20:26:11.961 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.053+0000 7fe4e9f9f700 1 -- 172.21.15.181:0/4069449169 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e41a2310 con 0x7fe4e4107cc0 2024-05-03T20:26:11.961 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.054+0000 7fe4e1ffb700 1 -- 172.21.15.181:0/4069449169 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fe4d8019800 con 0x7fe4e4107cc0 2024-05-03T20:26:11.961 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.054+0000 7fe4e9f9f700 1 -- 172.21.15.181:0/4069449169 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4e4064b00 con 0x7fe4e4107cc0 2024-05-03T20:26:11.961 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.054+0000 7fe4e1ffb700 1 --2- 172.21.15.181:0/4069449169 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7fe4d403e520 0x7fe4d40409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:11.961 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.054+0000 7fe4e1ffb700 1 -- 172.21.15.181:0/4069449169 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe4d8053970 con 0x7fe4e4107cc0 2024-05-03T20:26:11.962 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.055+0000 7fe4e3fff700 1 --2- 172.21.15.181:0/4069449169 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7fe4d403e520 0x7fe4d40409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:11.962 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.055+0000 7fe4e3fff700 1 --2- 172.21.15.181:0/4069449169 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7fe4d403e520 0x7fe4d40409e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0006fd0 tx=0x7fe4d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:11.962 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.060+0000 7fe4e1ffb700 1 -- 172.21.15.181:0/4069449169 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe4d8023390 con 0x7fe4e4107cc0 2024-05-03T20:26:11.962 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.262+0000 7fe4e9f9f700 1 -- 172.21.15.181:0/4069449169 --> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fe4e40638e0 con 0x7fe4d403e520 2024-05-03T20:26:11.962 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.359+0000 7fe4e1ffb700 1 -- 172.21.15.181:0/4069449169 <== mgr.14120 v2:172.21.15.181:6800/2389231557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe4e40638e0 con 0x7fe4d403e520 2024-05-03T20:26:11.962 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.361+0000 7fe4e9f9f700 1 -- 172.21.15.181:0/4069449169 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7fe4d403e520 msgr2=0x7fe4d40409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:11.962 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.361+0000 7fe4e9f9f700 1 --2- 172.21.15.181:0/4069449169 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7fe4d403e520 0x7fe4d40409e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0006fd0 tx=0x7fe4d0006e40 comp rx=0 tx=0).stop 2024-05-03T20:26:11.962 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.361+0000 7fe4e9f9f700 1 -- 172.21.15.181:0/4069449169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4e4107cc0 msgr2=0x7fe4e419ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:11.962 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.361+0000 7fe4e9f9f700 1 --2- 172.21.15.181:0/4069449169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4e4107cc0 0x7fe4e419ee50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8009fb0 tx=0x7fe4d8011940 comp rx=0 tx=0).stop 2024-05-03T20:26:11.962 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.361+0000 7fe4e9f9f700 1 -- 172.21.15.181:0/4069449169 shutdown_connections 2024-05-03T20:26:11.962 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.361+0000 7fe4e9f9f700 1 --2- 172.21.15.181:0/4069449169 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7fe4d403e520 0x7fe4d40409e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:11.962 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.361+0000 7fe4e9f9f700 1 --2- 172.21.15.181:0/4069449169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4e4107cc0 0x7fe4e419ee50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:11.962 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.361+0000 7fe4e9f9f700 1 -- 172.21.15.181:0/4069449169 >> 172.21.15.181:0/4069449169 conn(0x7fe4e4103280 msgr2=0x7fe4e4191080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:11.962 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.361+0000 7fe4e9f9f700 1 -- 172.21.15.181:0/4069449169 shutdown_connections 2024-05-03T20:26:11.963 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:11.361+0000 7fe4e9f9f700 1 -- 172.21.15.181:0/4069449169 wait complete. 2024-05-03T20:26:12.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:12 smithi181 ceph-mon[29801]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:26:12.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:12 smithi181 ceph-mon[29801]: Generating ssh key... 2024-05-03T20:26:12.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:12 smithi181 ceph-mon[29801]: from='mgr.14120 172.21.15.181:0/2717494325' entity='mgr.x' 2024-05-03T20:26:12.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:12 smithi181 ceph-mon[29801]: from='mgr.14120 172.21.15.181:0/2717494325' entity='mgr.x' 2024-05-03T20:26:13.176 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjGsDBfffgbt1QsmUENw/TRXe6U88W3TMV1ywiD495F1U/QGhgm/+nrec+Dj+gLiZ9OafbMUNLmpt+8uQ13L8Nyx6gK+msTLGd0Mhh7AhJBy3qUDaGkU18uCnsP1w7eJUsU6TBoD34n7TYGClM/oM+0vpxhPC3iRo+rZbkaC364JiNQWumxkRCp84GhOl2nTeFp8fq5V4pst5xkrTfshcjes0mkrnnDFD+Y91Giu1KFnWfQvT2olvg5ryddALxw3sVpi+7wjNQzBVHQHn2BbJo52PVMtCGI3qw8Z9WSwniQ86C4YNTqzU3cIwY6iZIPSMYaUvG8bNyNJzZUkdBv3MjbFJMfIOqQY1n64uzPR5xhKL5K4M8sbh0kzfU4gwSRHJ6mHIq7HzahvhiA7tFTvdKcMxP6BQsVQIdudN7RHKFAmrLcsf1y/+mefuuLhFqHfRB333X6kG+i8dSR6s9PkkjSsivNQDTNzBEjgyIC6snvxpSR1qYfSvEK+7DFrXIt30= ceph-1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:26:13.176 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.352+0000 7fc4c8f6a700 1 Processor -- start 2024-05-03T20:26:13.176 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.352+0000 7fc4c8f6a700 1 -- start start 2024-05-03T20:26:13.176 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.353+0000 7fc4c8f6a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4c4107e60 0x7fc4c4108280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:13.176 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.353+0000 7fc4c8f6a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c41087c0 con 0x7fc4c4107e60 2024-05-03T20:26:13.176 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.353+0000 7fc4c259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4c4107e60 0x7fc4c4108280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:13.176 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.353+0000 7fc4c259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4c4107e60 0x7fc4c4108280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57558/0 (socket says 172.21.15.181:57558) 2024-05-03T20:26:13.176 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.353+0000 7fc4c259c700 1 -- 172.21.15.181:0/2507549962 learned_addr learned my addr 172.21.15.181:0/2507549962 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:13.176 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.353+0000 7fc4c259c700 1 -- 172.21.15.181:0/2507549962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c4109050 con 0x7fc4c4107e60 2024-05-03T20:26:13.177 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.353+0000 7fc4c259c700 1 --2- 172.21.15.181:0/2507549962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4c4107e60 0x7fc4c4108280 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80098a0 tx=0x7fc4b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25972b9b392c9d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:13.177 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.353+0000 7fc4c159a700 1 -- 172.21.15.181:0/2507549962 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4b8004160 con 0x7fc4c4107e60 2024-05-03T20:26:13.177 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.353+0000 7fc4c159a700 1 -- 172.21.15.181:0/2507549962 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc4b8004300 con 0x7fc4c4107e60 2024-05-03T20:26:13.177 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.353+0000 7fc4c159a700 1 -- 172.21.15.181:0/2507549962 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4b8011af0 con 0x7fc4c4107e60 2024-05-03T20:26:13.177 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.354+0000 7fc4c8f6a700 1 -- 172.21.15.181:0/2507549962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4c4107e60 msgr2=0x7fc4c4108280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:13.177 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.354+0000 7fc4c8f6a700 1 --2- 172.21.15.181:0/2507549962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4c4107e60 0x7fc4c4108280 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80098a0 tx=0x7fc4b8009bb0 comp rx=0 tx=0).stop 2024-05-03T20:26:13.177 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.354+0000 7fc4c8f6a700 1 -- 172.21.15.181:0/2507549962 shutdown_connections 2024-05-03T20:26:13.177 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.354+0000 7fc4c8f6a700 1 --2- 172.21.15.181:0/2507549962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4c4107e60 0x7fc4c4108280 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:13.177 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.354+0000 7fc4c8f6a700 1 -- 172.21.15.181:0/2507549962 >> 172.21.15.181:0/2507549962 conn(0x7fc4c4103490 msgr2=0x7fc4c41058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:13.177 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.354+0000 7fc4c8f6a700 1 -- 172.21.15.181:0/2507549962 shutdown_connections 2024-05-03T20:26:13.177 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.354+0000 7fc4c8f6a700 1 -- 172.21.15.181:0/2507549962 wait complete. 2024-05-03T20:26:13.177 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.355+0000 7fc4c8f6a700 1 Processor -- start 2024-05-03T20:26:13.177 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.355+0000 7fc4c8f6a700 1 -- start start 2024-05-03T20:26:13.177 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.355+0000 7fc4c8f6a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4c4107e60 0x7fc4c419bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:13.177 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.355+0000 7fc4c8f6a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c4108940 con 0x7fc4c4107e60 2024-05-03T20:26:13.177 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.355+0000 7fc4c259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4c4107e60 0x7fc4c419bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:13.178 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.355+0000 7fc4c259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4c4107e60 0x7fc4c419bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57564/0 (socket says 172.21.15.181:57564) 2024-05-03T20:26:13.178 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.355+0000 7fc4c259c700 1 -- 172.21.15.181:0/2960690221 learned_addr learned my addr 172.21.15.181:0/2960690221 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:13.178 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.355+0000 7fc4c259c700 1 -- 172.21.15.181:0/2960690221 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c419c2d0 con 0x7fc4c4107e60 2024-05-03T20:26:13.178 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.356+0000 7fc4c259c700 1 --2- 172.21.15.181:0/2960690221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4c4107e60 0x7fc4c419bd90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8003f40 tx=0x7fc4b8011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:13.178 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.357+0000 7fc4b37fe700 1 -- 172.21.15.181:0/2960690221 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4b8003bf0 con 0x7fc4c4107e60 2024-05-03T20:26:13.178 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.357+0000 7fc4b37fe700 1 -- 172.21.15.181:0/2960690221 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc4b8020070 con 0x7fc4c4107e60 2024-05-03T20:26:13.178 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.357+0000 7fc4b37fe700 1 -- 172.21.15.181:0/2960690221 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4b801b6b0 con 0x7fc4c4107e60 2024-05-03T20:26:13.178 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.357+0000 7fc4c8f6a700 1 -- 172.21.15.181:0/2960690221 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c419c560 con 0x7fc4c4107e60 2024-05-03T20:26:13.178 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.357+0000 7fc4c8f6a700 1 -- 172.21.15.181:0/2960690221 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c407bfb0 con 0x7fc4c4107e60 2024-05-03T20:26:13.178 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.358+0000 7fc4b37fe700 1 -- 172.21.15.181:0/2960690221 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fc4b801b8d0 con 0x7fc4c4107e60 2024-05-03T20:26:13.178 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.358+0000 7fc4b37fe700 1 --2- 172.21.15.181:0/2960690221 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7fc4ac03e580 0x7fc4ac040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:13.178 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.358+0000 7fc4b37fe700 1 -- 172.21.15.181:0/2960690221 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc4b8056fb0 con 0x7fc4c4107e60 2024-05-03T20:26:13.178 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.358+0000 7fc4c8f6a700 1 -- 172.21.15.181:0/2960690221 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4c4064b00 con 0x7fc4c4107e60 2024-05-03T20:26:13.178 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.358+0000 7fc4c1d9b700 1 --2- 172.21.15.181:0/2960690221 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7fc4ac03e580 0x7fc4ac040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:13.179 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.359+0000 7fc4c1d9b700 1 --2- 172.21.15.181:0/2960690221 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7fc4ac03e580 0x7fc4ac040a40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4006fd0 tx=0x7fc4b4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:13.179 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.363+0000 7fc4b37fe700 1 -- 172.21.15.181:0/2960690221 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc4b802c2f0 con 0x7fc4c4107e60 2024-05-03T20:26:13.179 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.573+0000 7fc4c8f6a700 1 -- 172.21.15.181:0/2960690221 --> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fc4c4104610 con 0x7fc4ac03e580 2024-05-03T20:26:13.179 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.573+0000 7fc4b37fe700 1 -- 172.21.15.181:0/2960690221 <== mgr.14120 v2:172.21.15.181:6800/2389231557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fc4c4104610 con 0x7fc4ac03e580 2024-05-03T20:26:13.179 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.575+0000 7fc4c8f6a700 1 -- 172.21.15.181:0/2960690221 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7fc4ac03e580 msgr2=0x7fc4ac040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:13.179 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.575+0000 7fc4c8f6a700 1 --2- 172.21.15.181:0/2960690221 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7fc4ac03e580 0x7fc4ac040a40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4006fd0 tx=0x7fc4b4006e40 comp rx=0 tx=0).stop 2024-05-03T20:26:13.179 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.575+0000 7fc4c8f6a700 1 -- 172.21.15.181:0/2960690221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4c4107e60 msgr2=0x7fc4c419bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:13.179 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.575+0000 7fc4c8f6a700 1 --2- 172.21.15.181:0/2960690221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4c4107e60 0x7fc4c419bd90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8003f40 tx=0x7fc4b8011c20 comp rx=0 tx=0).stop 2024-05-03T20:26:13.179 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.576+0000 7fc4c8f6a700 1 -- 172.21.15.181:0/2960690221 shutdown_connections 2024-05-03T20:26:13.179 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.576+0000 7fc4c8f6a700 1 --2- 172.21.15.181:0/2960690221 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7fc4ac03e580 0x7fc4ac040a40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:13.179 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.576+0000 7fc4c8f6a700 1 --2- 172.21.15.181:0/2960690221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4c4107e60 0x7fc4c419bd90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:13.179 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.576+0000 7fc4c8f6a700 1 -- 172.21.15.181:0/2960690221 >> 172.21.15.181:0/2960690221 conn(0x7fc4c4103490 msgr2=0x7fc4c4103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:13.179 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.576+0000 7fc4c8f6a700 1 -- 172.21.15.181:0/2960690221 shutdown_connections 2024-05-03T20:26:13.179 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:12.576+0000 7fc4c8f6a700 1 -- 172.21.15.181:0/2960690221 wait complete. 2024-05-03T20:26:13.179 INFO:teuthology.orchestra.run.smithi181.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-03T20:26:13.180 INFO:teuthology.orchestra.run.smithi181.stdout:Adding key to root@localhost authorized_keys... 2024-05-03T20:26:13.180 INFO:teuthology.orchestra.run.smithi181.stdout:Adding host smithi181... 2024-05-03T20:26:13.807 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:13 smithi181 ceph-mon[29801]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:26:14.651 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:14 smithi181 ceph-mon[29801]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi181", "addr": "172.21.15.181", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:26:14.651 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:14 smithi181 ceph-mon[29801]: Deploying cephadm binary to smithi181 2024-05-03T20:26:18.517 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout Added host 'smithi181' with addr '172.21.15.181' 2024-05-03T20:26:18.517 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.591+0000 7f4540057700 1 Processor -- start 2024-05-03T20:26:18.517 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.591+0000 7f4540057700 1 -- start start 2024-05-03T20:26:18.517 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.592+0000 7f4540057700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4538109a20 0x7f453810be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:18.517 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.592+0000 7f4540057700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f453807ce50 con 0x7f4538109a20 2024-05-03T20:26:18.517 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.592+0000 7f453ddf3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4538109a20 0x7f453810be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:18.517 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.592+0000 7f453ddf3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4538109a20 0x7f453810be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57570/0 (socket says 172.21.15.181:57570) 2024-05-03T20:26:18.517 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.592+0000 7f453ddf3700 1 -- 172.21.15.181:0/3549626927 learned_addr learned my addr 172.21.15.181:0/3549626927 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:18.517 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.592+0000 7f453ddf3700 1 -- 172.21.15.181:0/3549626927 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453810c350 con 0x7f4538109a20 2024-05-03T20:26:18.517 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.592+0000 7f453ddf3700 1 --2- 172.21.15.181:0/3549626927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4538109a20 0x7f453810be10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f45340098a0 tx=0x7f4534009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db70bb309e63a0bf server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:18.517 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.592+0000 7f453cdf1700 1 -- 172.21.15.181:0/3549626927 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4534004160 con 0x7f4538109a20 2024-05-03T20:26:18.518 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.592+0000 7f453cdf1700 1 -- 172.21.15.181:0/3549626927 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4534004300 con 0x7f4538109a20 2024-05-03T20:26:18.518 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.592+0000 7f453cdf1700 1 -- 172.21.15.181:0/3549626927 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4534011af0 con 0x7f4538109a20 2024-05-03T20:26:18.518 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.593+0000 7f4540057700 1 -- 172.21.15.181:0/3549626927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4538109a20 msgr2=0x7f453810be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:18.518 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.593+0000 7f4540057700 1 --2- 172.21.15.181:0/3549626927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4538109a20 0x7f453810be10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f45340098a0 tx=0x7f4534009bb0 comp rx=0 tx=0).stop 2024-05-03T20:26:18.518 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.593+0000 7f4540057700 1 -- 172.21.15.181:0/3549626927 shutdown_connections 2024-05-03T20:26:18.518 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.593+0000 7f4540057700 1 --2- 172.21.15.181:0/3549626927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4538109a20 0x7f453810be10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:18.518 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.593+0000 7f4540057700 1 -- 172.21.15.181:0/3549626927 >> 172.21.15.181:0/3549626927 conn(0x7f4538103430 msgr2=0x7f4538105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:18.518 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.593+0000 7f4540057700 1 -- 172.21.15.181:0/3549626927 shutdown_connections 2024-05-03T20:26:18.518 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.593+0000 7f4540057700 1 -- 172.21.15.181:0/3549626927 wait complete. 2024-05-03T20:26:18.518 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.594+0000 7f4540057700 1 Processor -- start 2024-05-03T20:26:18.518 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.594+0000 7f4540057700 1 -- start start 2024-05-03T20:26:18.518 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.594+0000 7f4540057700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4538109a20 0x7f453819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:18.518 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.594+0000 7f4540057700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f453810d2a0 con 0x7f4538109a20 2024-05-03T20:26:18.518 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.594+0000 7f453ddf3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4538109a20 0x7f453819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:18.518 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.594+0000 7f453ddf3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4538109a20 0x7f453819cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57584/0 (socket says 172.21.15.181:57584) 2024-05-03T20:26:18.519 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.594+0000 7f453ddf3700 1 -- 172.21.15.181:0/3395568342 learned_addr learned my addr 172.21.15.181:0/3395568342 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:18.519 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.594+0000 7f453ddf3700 1 -- 172.21.15.181:0/3395568342 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453819d330 con 0x7f4538109a20 2024-05-03T20:26:18.519 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.594+0000 7f453ddf3700 1 --2- 172.21.15.181:0/3395568342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4538109a20 0x7f453819cdf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4534009fb0 tx=0x7f4534011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:18.519 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.595+0000 7f452effd700 1 -- 172.21.15.181:0/3395568342 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45340036a0 con 0x7f4538109a20 2024-05-03T20:26:18.519 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.595+0000 7f4540057700 1 -- 172.21.15.181:0/3395568342 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f453819d5c0 con 0x7f4538109a20 2024-05-03T20:26:18.519 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.595+0000 7f452effd700 1 -- 172.21.15.181:0/3395568342 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f453401a430 con 0x7f4538109a20 2024-05-03T20:26:18.519 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.595+0000 7f452effd700 1 -- 172.21.15.181:0/3395568342 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4534019550 con 0x7f4538109a20 2024-05-03T20:26:18.519 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.595+0000 7f4540057700 1 -- 172.21.15.181:0/3395568342 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45381a02b0 con 0x7f4538109a20 2024-05-03T20:26:18.519 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.596+0000 7f452effd700 1 -- 172.21.15.181:0/3395568342 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f4534019850 con 0x7f4538109a20 2024-05-03T20:26:18.519 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.596+0000 7f4540057700 1 -- 172.21.15.181:0/3395568342 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4538064b00 con 0x7f4538109a20 2024-05-03T20:26:18.519 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.596+0000 7f452effd700 1 --2- 172.21.15.181:0/3395568342 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f452403e4e0 0x7f45240409a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:18.519 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.596+0000 7f452effd700 1 -- 172.21.15.181:0/3395568342 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f45340539d0 con 0x7f4538109a20 2024-05-03T20:26:18.519 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.596+0000 7f453d5f2700 1 --2- 172.21.15.181:0/3395568342 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f452403e4e0 0x7f45240409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:18.519 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.596+0000 7f453d5f2700 1 --2- 172.21.15.181:0/3395568342 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f452403e4e0 0x7f45240409a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4528006fd0 tx=0x7f4528006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:18.519 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.601+0000 7f452effd700 1 -- 172.21.15.181:0/3395568342 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f453401a5e0 con 0x7f4538109a20 2024-05-03T20:26:18.520 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:13.807+0000 7f4540057700 1 -- 172.21.15.181:0/3395568342 --> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi181", "addr": "172.21.15.181", "target": ["mon-mgr", ""]}) -- 0x7f4538104630 con 0x7f452403e4e0 2024-05-03T20:26:18.520 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:17.857+0000 7f452effd700 1 -- 172.21.15.181:0/3395568342 <== mgr.14120 v2:172.21.15.181:6800/2389231557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f4538104630 con 0x7f452403e4e0 2024-05-03T20:26:18.520 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:17.860+0000 7f4540057700 1 -- 172.21.15.181:0/3395568342 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f452403e4e0 msgr2=0x7f45240409a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:18.520 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:17.860+0000 7f4540057700 1 --2- 172.21.15.181:0/3395568342 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f452403e4e0 0x7f45240409a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4528006fd0 tx=0x7f4528006e40 comp rx=0 tx=0).stop 2024-05-03T20:26:18.520 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:17.860+0000 7f4540057700 1 -- 172.21.15.181:0/3395568342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4538109a20 msgr2=0x7f453819cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:18.520 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:17.860+0000 7f4540057700 1 --2- 172.21.15.181:0/3395568342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4538109a20 0x7f453819cdf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4534009fb0 tx=0x7f4534011940 comp rx=0 tx=0).stop 2024-05-03T20:26:18.520 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:17.860+0000 7f4540057700 1 -- 172.21.15.181:0/3395568342 shutdown_connections 2024-05-03T20:26:18.520 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:17.860+0000 7f4540057700 1 --2- 172.21.15.181:0/3395568342 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f452403e4e0 0x7f45240409a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:18.520 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:17.860+0000 7f4540057700 1 --2- 172.21.15.181:0/3395568342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4538109a20 0x7f453819cdf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:18.520 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:17.860+0000 7f4540057700 1 -- 172.21.15.181:0/3395568342 >> 172.21.15.181:0/3395568342 conn(0x7f4538103430 msgr2=0x7f4538103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:18.520 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:17.860+0000 7f4540057700 1 -- 172.21.15.181:0/3395568342 shutdown_connections 2024-05-03T20:26:18.520 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:17.860+0000 7f4540057700 1 -- 172.21.15.181:0/3395568342 wait complete. 2024-05-03T20:26:18.520 INFO:teuthology.orchestra.run.smithi181.stdout:Deploying unmanaged mon service... 2024-05-03T20:26:19.071 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:18 smithi181 ceph-mon[29801]: from='mgr.14120 172.21.15.181:0/2717494325' entity='mgr.x' 2024-05-03T20:26:19.071 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:18 smithi181 ceph-mon[29801]: Added host smithi181 2024-05-03T20:26:19.071 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:18 smithi181 ceph-mon[29801]: from='mgr.14120 172.21.15.181:0/2717494325' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:26:20.052 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-03T20:26:20.052 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.173+0000 7f11f4d9a700 1 Processor -- start 2024-05-03T20:26:20.052 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.173+0000 7f11f4d9a700 1 -- start start 2024-05-03T20:26:20.052 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.173+0000 7f11f4d9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11f0108f10 0x7f11f0109350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:20.052 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.173+0000 7f11f4d9a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f11f0109890 con 0x7f11f0108f10 2024-05-03T20:26:20.053 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.174+0000 7f11ee59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11f0108f10 0x7f11f0109350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:20.053 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.174+0000 7f11ee59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11f0108f10 0x7f11f0109350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54032/0 (socket says 172.21.15.181:54032) 2024-05-03T20:26:20.053 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.174+0000 7f11ee59c700 1 -- 172.21.15.181:0/3351206833 learned_addr learned my addr 172.21.15.181:0/3351206833 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:20.053 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.174+0000 7f11ee59c700 1 -- 172.21.15.181:0/3351206833 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f010a0f0 con 0x7f11f0108f10 2024-05-03T20:26:20.053 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.174+0000 7f11ee59c700 1 --2- 172.21.15.181:0/3351206833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11f0108f10 0x7f11f0109350 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f11e00098a0 tx=0x7f11e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2034065eee4cd725 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:20.053 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.174+0000 7f11ed59a700 1 -- 172.21.15.181:0/3351206833 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11e0004160 con 0x7f11f0108f10 2024-05-03T20:26:20.053 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.174+0000 7f11ed59a700 1 -- 172.21.15.181:0/3351206833 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f11e0004300 con 0x7f11f0108f10 2024-05-03T20:26:20.053 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.174+0000 7f11ed59a700 1 -- 172.21.15.181:0/3351206833 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11e0011af0 con 0x7f11f0108f10 2024-05-03T20:26:20.053 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.175+0000 7f11f4d9a700 1 -- 172.21.15.181:0/3351206833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11f0108f10 msgr2=0x7f11f0109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:20.053 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.175+0000 7f11f4d9a700 1 --2- 172.21.15.181:0/3351206833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11f0108f10 0x7f11f0109350 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f11e00098a0 tx=0x7f11e0009bb0 comp rx=0 tx=0).stop 2024-05-03T20:26:20.053 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.175+0000 7f11f4d9a700 1 -- 172.21.15.181:0/3351206833 shutdown_connections 2024-05-03T20:26:20.053 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.175+0000 7f11f4d9a700 1 --2- 172.21.15.181:0/3351206833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11f0108f10 0x7f11f0109350 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:20.053 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.175+0000 7f11f4d9a700 1 -- 172.21.15.181:0/3351206833 >> 172.21.15.181:0/3351206833 conn(0x7f11f0104520 msgr2=0x7f11f0106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:20.053 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.175+0000 7f11f4d9a700 1 -- 172.21.15.181:0/3351206833 shutdown_connections 2024-05-03T20:26:20.054 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.175+0000 7f11f4d9a700 1 -- 172.21.15.181:0/3351206833 wait complete. 2024-05-03T20:26:20.054 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.176+0000 7f11f4d9a700 1 Processor -- start 2024-05-03T20:26:20.054 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.176+0000 7f11f4d9a700 1 -- start start 2024-05-03T20:26:20.054 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.176+0000 7f11f4d9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11f0108f10 0x7f11f019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:20.054 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.176+0000 7f11f4d9a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f11f010af10 con 0x7f11f0108f10 2024-05-03T20:26:20.054 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.176+0000 7f11ee59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11f0108f10 0x7f11f019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:20.054 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.176+0000 7f11ee59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11f0108f10 0x7f11f019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54034/0 (socket says 172.21.15.181:54034) 2024-05-03T20:26:20.054 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.176+0000 7f11ee59c700 1 -- 172.21.15.181:0/1029725650 learned_addr learned my addr 172.21.15.181:0/1029725650 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:20.054 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.176+0000 7f11ee59c700 1 -- 172.21.15.181:0/1029725650 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f019d2e0 con 0x7f11f0108f10 2024-05-03T20:26:20.054 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.176+0000 7f11ee59c700 1 --2- 172.21.15.181:0/1029725650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11f0108f10 0x7f11f019cda0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f11e0003f40 tx=0x7f11e0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:20.054 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.176+0000 7f11df7fe700 1 -- 172.21.15.181:0/1029725650 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11e00038b0 con 0x7f11f0108f10 2024-05-03T20:26:20.054 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.176+0000 7f11df7fe700 1 -- 172.21.15.181:0/1029725650 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f11e0003a50 con 0x7f11f0108f10 2024-05-03T20:26:20.054 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.176+0000 7f11f4d9a700 1 -- 172.21.15.181:0/1029725650 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11f019d570 con 0x7f11f0108f10 2024-05-03T20:26:20.054 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.176+0000 7f11df7fe700 1 -- 172.21.15.181:0/1029725650 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11e001a5c0 con 0x7f11f0108f10 2024-05-03T20:26:20.055 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.176+0000 7f11f4d9a700 1 -- 172.21.15.181:0/1029725650 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11f01a0220 con 0x7f11f0108f10 2024-05-03T20:26:20.055 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.177+0000 7f11df7fe700 1 -- 172.21.15.181:0/1029725650 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f11e0024430 con 0x7f11f0108f10 2024-05-03T20:26:20.055 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.177+0000 7f11f4d9a700 1 -- 172.21.15.181:0/1029725650 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11f0064b00 con 0x7f11f0108f10 2024-05-03T20:26:20.055 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.177+0000 7f11df7fe700 1 --2- 172.21.15.181:0/1029725650 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f11d803e530 0x7f11d80409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:20.055 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.177+0000 7f11df7fe700 1 -- 172.21.15.181:0/1029725650 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f11e0053a40 con 0x7f11f0108f10 2024-05-03T20:26:20.055 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.178+0000 7f11edd9b700 1 --2- 172.21.15.181:0/1029725650 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f11d803e530 0x7f11d80409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:20.055 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.178+0000 7f11edd9b700 1 --2- 172.21.15.181:0/1029725650 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f11d803e530 0x7f11d80409f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f11e4006fd0 tx=0x7f11e4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:20.055 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.183+0000 7f11df7fe700 1 -- 172.21.15.181:0/1029725650 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f11e0055e20 con 0x7f11f0108f10 2024-05-03T20:26:20.055 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.412+0000 7f11f4d9a700 1 -- 172.21.15.181:0/1029725650 --> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f11f0105690 con 0x7f11d803e530 2024-05-03T20:26:20.055 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.418+0000 7f11df7fe700 1 -- 172.21.15.181:0/1029725650 <== mgr.14120 v2:172.21.15.181:6800/2389231557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f11f0105690 con 0x7f11d803e530 2024-05-03T20:26:20.055 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.420+0000 7f11f4d9a700 1 -- 172.21.15.181:0/1029725650 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f11d803e530 msgr2=0x7f11d80409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:20.055 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.420+0000 7f11f4d9a700 1 --2- 172.21.15.181:0/1029725650 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f11d803e530 0x7f11d80409f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f11e4006fd0 tx=0x7f11e4006e40 comp rx=0 tx=0).stop 2024-05-03T20:26:20.055 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.420+0000 7f11f4d9a700 1 -- 172.21.15.181:0/1029725650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11f0108f10 msgr2=0x7f11f019cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:20.056 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.420+0000 7f11f4d9a700 1 --2- 172.21.15.181:0/1029725650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11f0108f10 0x7f11f019cda0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f11e0003f40 tx=0x7f11e0011890 comp rx=0 tx=0).stop 2024-05-03T20:26:20.056 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.420+0000 7f11f4d9a700 1 -- 172.21.15.181:0/1029725650 shutdown_connections 2024-05-03T20:26:20.056 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.420+0000 7f11f4d9a700 1 --2- 172.21.15.181:0/1029725650 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f11d803e530 0x7f11d80409f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:20.056 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.420+0000 7f11f4d9a700 1 --2- 172.21.15.181:0/1029725650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11f0108f10 0x7f11f019cda0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:20.056 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.420+0000 7f11f4d9a700 1 -- 172.21.15.181:0/1029725650 >> 172.21.15.181:0/1029725650 conn(0x7f11f0104520 msgr2=0x7f11f0104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:20.056 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.420+0000 7f11f4d9a700 1 -- 172.21.15.181:0/1029725650 shutdown_connections 2024-05-03T20:26:20.056 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:19.420+0000 7f11f4d9a700 1 -- 172.21.15.181:0/1029725650 wait complete. 2024-05-03T20:26:20.056 INFO:teuthology.orchestra.run.smithi181.stdout:Deploying unmanaged mgr service... 2024-05-03T20:26:20.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:20 smithi181 ceph-mon[29801]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:26:20.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:20 smithi181 ceph-mon[29801]: Saving service mon spec with placement count:5 2024-05-03T20:26:20.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:20 smithi181 ceph-mon[29801]: from='mgr.14120 172.21.15.181:0/2717494325' entity='mgr.x' 2024-05-03T20:26:22.130 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:21 smithi181 ceph-mon[29801]: from='mgr.14120 172.21.15.181:0/2717494325' entity='mgr.x' 2024-05-03T20:26:22.130 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:21 smithi181 ceph-mon[29801]: from='mgr.14120 172.21.15.181:0/2717494325' entity='mgr.x' 2024-05-03T20:26:22.131 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:21 smithi181 ceph-mon[29801]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:26:22.131 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:21 smithi181 ceph-mon[29801]: Saving service mgr spec with placement count:2 2024-05-03T20:26:22.131 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:21 smithi181 ceph-mon[29801]: from='mgr.14120 172.21.15.181:0/2717494325' entity='mgr.x' 2024-05-03T20:26:22.138 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-03T20:26:22.138 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.279+0000 7f0d6b18c700 1 Processor -- start 2024-05-03T20:26:22.138 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.279+0000 7f0d6b18c700 1 -- start start 2024-05-03T20:26:22.138 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.280+0000 7f0d6b18c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d64107860 0x7f0d64107c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:22.138 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.280+0000 7f0d6b18c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6407ce10 con 0x7f0d64107860 2024-05-03T20:26:22.138 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.280+0000 7f0d68f28700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d64107860 0x7f0d64107c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:22.138 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.280+0000 7f0d68f28700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d64107860 0x7f0d64107c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54036/0 (socket says 172.21.15.181:54036) 2024-05-03T20:26:22.138 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.280+0000 7f0d68f28700 1 -- 172.21.15.181:0/2654543987 learned_addr learned my addr 172.21.15.181:0/2654543987 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:22.138 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.280+0000 7f0d68f28700 1 -- 172.21.15.181:0/2654543987 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d641081c0 con 0x7f0d64107860 2024-05-03T20:26:22.139 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.280+0000 7f0d68f28700 1 --2- 172.21.15.181:0/2654543987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d64107860 0x7f0d64107c80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0d580098a0 tx=0x7f0d58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ef972cdd9e101c server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:22.139 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.280+0000 7f0d637fe700 1 -- 172.21.15.181:0/2654543987 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d58004160 con 0x7f0d64107860 2024-05-03T20:26:22.139 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.280+0000 7f0d637fe700 1 -- 172.21.15.181:0/2654543987 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d58004300 con 0x7f0d64107860 2024-05-03T20:26:22.139 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.280+0000 7f0d637fe700 1 -- 172.21.15.181:0/2654543987 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d58011a20 con 0x7f0d64107860 2024-05-03T20:26:22.139 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.281+0000 7f0d6b18c700 1 -- 172.21.15.181:0/2654543987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d64107860 msgr2=0x7f0d64107c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:22.139 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.281+0000 7f0d6b18c700 1 --2- 172.21.15.181:0/2654543987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d64107860 0x7f0d64107c80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0d580098a0 tx=0x7f0d58009bb0 comp rx=0 tx=0).stop 2024-05-03T20:26:22.139 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.281+0000 7f0d6b18c700 1 -- 172.21.15.181:0/2654543987 shutdown_connections 2024-05-03T20:26:22.139 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.281+0000 7f0d6b18c700 1 --2- 172.21.15.181:0/2654543987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d64107860 0x7f0d64107c80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:22.139 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.281+0000 7f0d6b18c700 1 -- 172.21.15.181:0/2654543987 >> 172.21.15.181:0/2654543987 conn(0x7f0d64103440 msgr2=0x7f0d64105860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:22.139 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.281+0000 7f0d6b18c700 1 -- 172.21.15.181:0/2654543987 shutdown_connections 2024-05-03T20:26:22.139 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.281+0000 7f0d6b18c700 1 -- 172.21.15.181:0/2654543987 wait complete. 2024-05-03T20:26:22.139 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.282+0000 7f0d6b18c700 1 Processor -- start 2024-05-03T20:26:22.139 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.282+0000 7f0d6b18c700 1 -- start start 2024-05-03T20:26:22.139 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.282+0000 7f0d6b18c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d64107860 0x7f0d6419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:22.140 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.282+0000 7f0d6b18c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0d64109230 con 0x7f0d64107860 2024-05-03T20:26:22.140 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.282+0000 7f0d68f28700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d64107860 0x7f0d6419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:22.140 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.282+0000 7f0d68f28700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d64107860 0x7f0d6419efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54048/0 (socket says 172.21.15.181:54048) 2024-05-03T20:26:22.140 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.282+0000 7f0d68f28700 1 -- 172.21.15.181:0/2146514244 learned_addr learned my addr 172.21.15.181:0/2146514244 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:22.140 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.282+0000 7f0d68f28700 1 -- 172.21.15.181:0/2146514244 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d6419f520 con 0x7f0d64107860 2024-05-03T20:26:22.140 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.282+0000 7f0d68f28700 1 --2- 172.21.15.181:0/2146514244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d64107860 0x7f0d6419efe0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0d58009520 tx=0x7f0d58011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:22.140 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.283+0000 7f0d61ffb700 1 -- 172.21.15.181:0/2146514244 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d58003b70 con 0x7f0d64107860 2024-05-03T20:26:22.140 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.283+0000 7f0d61ffb700 1 -- 172.21.15.181:0/2146514244 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d5801b8f0 con 0x7f0d64107860 2024-05-03T20:26:22.140 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.283+0000 7f0d6b18c700 1 -- 172.21.15.181:0/2146514244 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d6419f7b0 con 0x7f0d64107860 2024-05-03T20:26:22.140 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.283+0000 7f0d61ffb700 1 -- 172.21.15.181:0/2146514244 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d58019800 con 0x7f0d64107860 2024-05-03T20:26:22.140 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.283+0000 7f0d6b18c700 1 -- 172.21.15.181:0/2146514244 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d641a24a0 con 0x7f0d64107860 2024-05-03T20:26:22.140 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.284+0000 7f0d61ffb700 1 -- 172.21.15.181:0/2146514244 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f0d58019ab0 con 0x7f0d64107860 2024-05-03T20:26:22.140 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.284+0000 7f0d6b18c700 1 -- 172.21.15.181:0/2146514244 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d64064b00 con 0x7f0d64107860 2024-05-03T20:26:22.140 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.284+0000 7f0d61ffb700 1 --2- 172.21.15.181:0/2146514244 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f0d5403e530 0x7f0d540409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:22.140 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.284+0000 7f0d61ffb700 1 -- 172.21.15.181:0/2146514244 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0d580540c0 con 0x7f0d64107860 2024-05-03T20:26:22.140 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.284+0000 7f0d63fff700 1 --2- 172.21.15.181:0/2146514244 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f0d5403e530 0x7f0d540409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:22.141 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.285+0000 7f0d63fff700 1 --2- 172.21.15.181:0/2146514244 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f0d5403e530 0x7f0d540409f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0d50006fd0 tx=0x7f0d50006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:22.141 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.289+0000 7f0d61ffb700 1 -- 172.21.15.181:0/2146514244 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0d5801f030 con 0x7f0d64107860 2024-05-03T20:26:22.141 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.519+0000 7f0d6b18c700 1 -- 172.21.15.181:0/2146514244 --> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f0d6402e7d0 con 0x7f0d5403e530 2024-05-03T20:26:22.141 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.526+0000 7f0d61ffb700 1 -- 172.21.15.181:0/2146514244 <== mgr.14120 v2:172.21.15.181:6800/2389231557 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0d6402e7d0 con 0x7f0d5403e530 2024-05-03T20:26:22.141 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.529+0000 7f0d6b18c700 1 -- 172.21.15.181:0/2146514244 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f0d5403e530 msgr2=0x7f0d540409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:22.141 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.529+0000 7f0d6b18c700 1 --2- 172.21.15.181:0/2146514244 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f0d5403e530 0x7f0d540409f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0d50006fd0 tx=0x7f0d50006e40 comp rx=0 tx=0).stop 2024-05-03T20:26:22.141 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.529+0000 7f0d6b18c700 1 -- 172.21.15.181:0/2146514244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d64107860 msgr2=0x7f0d6419efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:22.141 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.529+0000 7f0d6b18c700 1 --2- 172.21.15.181:0/2146514244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d64107860 0x7f0d6419efe0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0d58009520 tx=0x7f0d58011b10 comp rx=0 tx=0).stop 2024-05-03T20:26:22.141 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.529+0000 7f0d6b18c700 1 -- 172.21.15.181:0/2146514244 shutdown_connections 2024-05-03T20:26:22.141 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.529+0000 7f0d6b18c700 1 --2- 172.21.15.181:0/2146514244 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f0d5403e530 0x7f0d540409f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:22.141 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.529+0000 7f0d6b18c700 1 --2- 172.21.15.181:0/2146514244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d64107860 0x7f0d6419efe0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:22.141 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.529+0000 7f0d6b18c700 1 -- 172.21.15.181:0/2146514244 >> 172.21.15.181:0/2146514244 conn(0x7f0d64103440 msgr2=0x7f0d64195c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:22.141 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.529+0000 7f0d6b18c700 1 -- 172.21.15.181:0/2146514244 shutdown_connections 2024-05-03T20:26:22.141 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:21.529+0000 7f0d6b18c700 1 -- 172.21.15.181:0/2146514244 wait complete. 2024-05-03T20:26:23.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:23 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/1774984590' entity='client.admin' 2024-05-03T20:26:23.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:23 smithi181 ceph-mon[29801]: from='mgr.14120 172.21.15.181:0/2717494325' entity='mgr.x' 2024-05-03T20:26:23.990 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.552+0000 7f97debf4700 1 Processor -- start 2024-05-03T20:26:23.990 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.552+0000 7f97debf4700 1 -- start start 2024-05-03T20:26:23.990 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.552+0000 7f97debf4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97d8108670 0x7f97d810aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:23.990 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.552+0000 7f97debf4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f97d810b030 con 0x7f97d8108670 2024-05-03T20:26:23.991 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.552+0000 7f97dc990700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97d8108670 0x7f97d810aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:23.991 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.552+0000 7f97dc990700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97d8108670 0x7f97d810aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54062/0 (socket says 172.21.15.181:54062) 2024-05-03T20:26:23.991 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.552+0000 7f97dc990700 1 -- 172.21.15.181:0/4249816610 learned_addr learned my addr 172.21.15.181:0/4249816610 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:23.991 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.552+0000 7f97dc990700 1 -- 172.21.15.181:0/4249816610 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d810b1b0 con 0x7f97d8108670 2024-05-03T20:26:23.991 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.553+0000 7f97dc990700 1 --2- 172.21.15.181:0/4249816610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97d8108670 0x7f97d810aa60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0098a0 tx=0x7f97cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33aee6837084a4d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:23.991 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.553+0000 7f97d77fe700 1 -- 172.21.15.181:0/4249816610 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97cc004160 con 0x7f97d8108670 2024-05-03T20:26:23.991 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.553+0000 7f97d77fe700 1 -- 172.21.15.181:0/4249816610 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97cc004300 con 0x7f97d8108670 2024-05-03T20:26:23.991 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.553+0000 7f97d77fe700 1 -- 172.21.15.181:0/4249816610 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97cc011a20 con 0x7f97d8108670 2024-05-03T20:26:23.991 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.553+0000 7f97debf4700 1 -- 172.21.15.181:0/4249816610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97d8108670 msgr2=0x7f97d810aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:23.991 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.553+0000 7f97debf4700 1 --2- 172.21.15.181:0/4249816610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97d8108670 0x7f97d810aa60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0098a0 tx=0x7f97cc009bb0 comp rx=0 tx=0).stop 2024-05-03T20:26:23.991 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.553+0000 7f97debf4700 1 -- 172.21.15.181:0/4249816610 shutdown_connections 2024-05-03T20:26:23.991 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.553+0000 7f97debf4700 1 --2- 172.21.15.181:0/4249816610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97d8108670 0x7f97d810aa60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:23.991 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.553+0000 7f97debf4700 1 -- 172.21.15.181:0/4249816610 >> 172.21.15.181:0/4249816610 conn(0x7f97d8102750 msgr2=0x7f97d8104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:23.991 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.553+0000 7f97debf4700 1 -- 172.21.15.181:0/4249816610 shutdown_connections 2024-05-03T20:26:23.992 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.553+0000 7f97debf4700 1 -- 172.21.15.181:0/4249816610 wait complete. 2024-05-03T20:26:23.992 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.554+0000 7f97debf4700 1 Processor -- start 2024-05-03T20:26:23.992 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.554+0000 7f97debf4700 1 -- start start 2024-05-03T20:26:23.992 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.554+0000 7f97debf4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97d8108670 0x7f97d819bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:23.992 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.554+0000 7f97debf4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f97d810c1b0 con 0x7f97d8108670 2024-05-03T20:26:23.992 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.554+0000 7f97dc990700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97d8108670 0x7f97d819bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:23.992 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.554+0000 7f97dc990700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97d8108670 0x7f97d819bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54068/0 (socket says 172.21.15.181:54068) 2024-05-03T20:26:23.992 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.554+0000 7f97dc990700 1 -- 172.21.15.181:0/1774984590 learned_addr learned my addr 172.21.15.181:0/1774984590 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:23.992 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.554+0000 7f97dc990700 1 -- 172.21.15.181:0/1774984590 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d819c140 con 0x7f97d8108670 2024-05-03T20:26:23.992 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.555+0000 7f97dc990700 1 --2- 172.21.15.181:0/1774984590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97d8108670 0x7f97d819bc00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0116e0 tx=0x7f97cc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:23.992 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.555+0000 7f97d5ffb700 1 -- 172.21.15.181:0/1774984590 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97cc003b20 con 0x7f97d8108670 2024-05-03T20:26:23.992 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.555+0000 7f97d5ffb700 1 -- 172.21.15.181:0/1774984590 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97cc01b8f0 con 0x7f97d8108670 2024-05-03T20:26:23.992 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.555+0000 7f97debf4700 1 -- 172.21.15.181:0/1774984590 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d819c430 con 0x7f97d8108670 2024-05-03T20:26:23.992 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.555+0000 7f97d5ffb700 1 -- 172.21.15.181:0/1774984590 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97cc0238c0 con 0x7f97d8108670 2024-05-03T20:26:23.992 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.555+0000 7f97debf4700 1 -- 172.21.15.181:0/1774984590 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d819f0c0 con 0x7f97d8108670 2024-05-03T20:26:23.993 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.556+0000 7f97d5ffb700 1 -- 172.21.15.181:0/1774984590 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f97cc023a60 con 0x7f97d8108670 2024-05-03T20:26:23.993 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.556+0000 7f97debf4700 1 -- 172.21.15.181:0/1774984590 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97d8064b00 con 0x7f97d8108670 2024-05-03T20:26:23.993 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.556+0000 7f97d5ffb700 1 --2- 172.21.15.181:0/1774984590 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f97c803e530 0x7f97c80409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:23.993 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.556+0000 7f97d5ffb700 1 -- 172.21.15.181:0/1774984590 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f97cc054010 con 0x7f97d8108670 2024-05-03T20:26:23.993 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.556+0000 7f97d7fff700 1 --2- 172.21.15.181:0/1774984590 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f97c803e530 0x7f97c80409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:23.993 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.556+0000 7f97d7fff700 1 --2- 172.21.15.181:0/1774984590 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f97c803e530 0x7f97c80409f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f97c4006fd0 tx=0x7f97c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:23.993 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.561+0000 7f97d5ffb700 1 -- 172.21.15.181:0/1774984590 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f97cc01f030 con 0x7f97d8108670 2024-05-03T20:26:23.993 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.780+0000 7f97debf4700 1 -- 172.21.15.181:0/1774984590 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f97d819f290 con 0x7f97d8108670 2024-05-03T20:26:23.993 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.783+0000 7f97d5ffb700 1 -- 172.21.15.181:0/1774984590 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f97cc023e00 con 0x7f97d8108670 2024-05-03T20:26:23.993 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.785+0000 7f97debf4700 1 -- 172.21.15.181:0/1774984590 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f97c803e530 msgr2=0x7f97c80409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:23.993 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.785+0000 7f97debf4700 1 --2- 172.21.15.181:0/1774984590 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f97c803e530 0x7f97c80409f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f97c4006fd0 tx=0x7f97c4006e40 comp rx=0 tx=0).stop 2024-05-03T20:26:23.993 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.785+0000 7f97debf4700 1 -- 172.21.15.181:0/1774984590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97d8108670 msgr2=0x7f97d819bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:23.993 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.785+0000 7f97debf4700 1 --2- 172.21.15.181:0/1774984590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97d8108670 0x7f97d819bc00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0116e0 tx=0x7f97cc011aa0 comp rx=0 tx=0).stop 2024-05-03T20:26:23.993 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.785+0000 7f97debf4700 1 -- 172.21.15.181:0/1774984590 shutdown_connections 2024-05-03T20:26:23.994 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.785+0000 7f97debf4700 1 --2- 172.21.15.181:0/1774984590 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f97c803e530 0x7f97c80409f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:23.994 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.785+0000 7f97debf4700 1 --2- 172.21.15.181:0/1774984590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97d8108670 0x7f97d819bc00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:23.994 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.785+0000 7f97debf4700 1 -- 172.21.15.181:0/1774984590 >> 172.21.15.181:0/1774984590 conn(0x7f97d8102750 msgr2=0x7f97d81031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:23.994 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.786+0000 7f97debf4700 1 -- 172.21.15.181:0/1774984590 shutdown_connections 2024-05-03T20:26:23.994 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:22.786+0000 7f97debf4700 1 -- 172.21.15.181:0/1774984590 wait complete. 2024-05-03T20:26:25.263 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.437+0000 7f08485e3700 1 Processor -- start 2024-05-03T20:26:25.263 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.437+0000 7f08485e3700 1 -- start start 2024-05-03T20:26:25.263 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.437+0000 7f08485e3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0840107860 0x7f0840107c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:25.263 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.437+0000 7f08485e3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f084007ce10 con 0x7f0840107860 2024-05-03T20:26:25.263 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.438+0000 7f084637f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0840107860 0x7f0840107c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:25.263 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.438+0000 7f084637f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0840107860 0x7f0840107c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54082/0 (socket says 172.21.15.181:54082) 2024-05-03T20:26:25.263 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.438+0000 7f084637f700 1 -- 172.21.15.181:0/3249409886 learned_addr learned my addr 172.21.15.181:0/3249409886 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:25.263 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.438+0000 7f084637f700 1 -- 172.21.15.181:0/3249409886 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08401081c0 con 0x7f0840107860 2024-05-03T20:26:25.263 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.438+0000 7f084637f700 1 --2- 172.21.15.181:0/3249409886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0840107860 0x7f0840107c80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0830009e80 tx=0x7f083000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9027f5501f9409 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:25.263 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.438+0000 7f084537d700 1 -- 172.21.15.181:0/3249409886 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0830017070 con 0x7f0840107860 2024-05-03T20:26:25.264 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.438+0000 7f084537d700 1 -- 172.21.15.181:0/3249409886 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f083000b8c0 con 0x7f0840107860 2024-05-03T20:26:25.264 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.438+0000 7f084537d700 1 -- 172.21.15.181:0/3249409886 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0830004490 con 0x7f0840107860 2024-05-03T20:26:25.264 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.439+0000 7f08485e3700 1 -- 172.21.15.181:0/3249409886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0840107860 msgr2=0x7f0840107c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:25.264 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.439+0000 7f08485e3700 1 --2- 172.21.15.181:0/3249409886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0840107860 0x7f0840107c80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0830009e80 tx=0x7f083000b330 comp rx=0 tx=0).stop 2024-05-03T20:26:25.264 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.439+0000 7f08485e3700 1 -- 172.21.15.181:0/3249409886 shutdown_connections 2024-05-03T20:26:25.264 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.439+0000 7f08485e3700 1 --2- 172.21.15.181:0/3249409886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0840107860 0x7f0840107c80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:25.264 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.439+0000 7f08485e3700 1 -- 172.21.15.181:0/3249409886 >> 172.21.15.181:0/3249409886 conn(0x7f0840103440 msgr2=0x7f0840105860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:25.264 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.439+0000 7f08485e3700 1 -- 172.21.15.181:0/3249409886 shutdown_connections 2024-05-03T20:26:25.264 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.439+0000 7f08485e3700 1 -- 172.21.15.181:0/3249409886 wait complete. 2024-05-03T20:26:25.264 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.439+0000 7f08485e3700 1 Processor -- start 2024-05-03T20:26:25.264 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.440+0000 7f08485e3700 1 -- start start 2024-05-03T20:26:25.264 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.440+0000 7f08485e3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0840107860 0x7f084019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:25.264 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.440+0000 7f08485e3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0840109230 con 0x7f0840107860 2024-05-03T20:26:25.264 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.440+0000 7f084637f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0840107860 0x7f084019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:25.264 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.440+0000 7f084637f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0840107860 0x7f084019efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54096/0 (socket says 172.21.15.181:54096) 2024-05-03T20:26:25.265 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.440+0000 7f084637f700 1 -- 172.21.15.181:0/3313776961 learned_addr learned my addr 172.21.15.181:0/3313776961 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:25.265 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.440+0000 7f084637f700 1 -- 172.21.15.181:0/3313776961 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084019f4e0 con 0x7f0840107860 2024-05-03T20:26:25.265 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.440+0000 7f084637f700 1 --2- 172.21.15.181:0/3313776961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0840107860 0x7f084019efa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f083000b7b0 tx=0x7f083000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:25.265 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.440+0000 7f083f7fe700 1 -- 172.21.15.181:0/3313776961 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0830020020 con 0x7f0840107860 2024-05-03T20:26:25.265 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.440+0000 7f083f7fe700 1 -- 172.21.15.181:0/3313776961 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0830003c60 con 0x7f0840107860 2024-05-03T20:26:25.265 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.440+0000 7f083f7fe700 1 -- 172.21.15.181:0/3313776961 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0830017040 con 0x7f0840107860 2024-05-03T20:26:25.265 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.440+0000 7f08485e3700 1 -- 172.21.15.181:0/3313776961 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f084019f770 con 0x7f0840107860 2024-05-03T20:26:25.265 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.440+0000 7f08485e3700 1 -- 172.21.15.181:0/3313776961 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08401a2460 con 0x7f0840107860 2024-05-03T20:26:25.265 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.441+0000 7f08485e3700 1 -- 172.21.15.181:0/3313776961 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0840064b00 con 0x7f0840107860 2024-05-03T20:26:25.265 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.441+0000 7f083f7fe700 1 -- 172.21.15.181:0/3313776961 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f0830023020 con 0x7f0840107860 2024-05-03T20:26:25.265 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.441+0000 7f083f7fe700 1 --2- 172.21.15.181:0/3313776961 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f082803e580 0x7f0828040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:25.265 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.441+0000 7f083f7fe700 1 -- 172.21.15.181:0/3313776961 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f083000f050 con 0x7f0840107860 2024-05-03T20:26:25.265 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.442+0000 7f0845b7e700 1 --2- 172.21.15.181:0/3313776961 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f082803e580 0x7f0828040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:25.266 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.442+0000 7f0845b7e700 1 --2- 172.21.15.181:0/3313776961 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f082803e580 0x7f0828040a40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0834006fd0 tx=0x7f0834006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:25.266 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.446+0000 7f083f7fe700 1 -- 172.21.15.181:0/3313776961 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f083002a020 con 0x7f0840107860 2024-05-03T20:26:25.266 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.672+0000 7f08485e3700 1 -- 172.21.15.181:0/3313776961 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f084002e8c0 con 0x7f0840107860 2024-05-03T20:26:25.266 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.674+0000 7f083f7fe700 1 -- 172.21.15.181:0/3313776961 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f083001a2d0 con 0x7f0840107860 2024-05-03T20:26:25.266 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.677+0000 7f08485e3700 1 -- 172.21.15.181:0/3313776961 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f082803e580 msgr2=0x7f0828040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:25.266 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.677+0000 7f08485e3700 1 --2- 172.21.15.181:0/3313776961 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f082803e580 0x7f0828040a40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0834006fd0 tx=0x7f0834006e40 comp rx=0 tx=0).stop 2024-05-03T20:26:25.266 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.677+0000 7f08485e3700 1 -- 172.21.15.181:0/3313776961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0840107860 msgr2=0x7f084019efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:25.266 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.677+0000 7f08485e3700 1 --2- 172.21.15.181:0/3313776961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0840107860 0x7f084019efa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f083000b7b0 tx=0x7f083000bd50 comp rx=0 tx=0).stop 2024-05-03T20:26:25.266 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.677+0000 7f08485e3700 1 -- 172.21.15.181:0/3313776961 shutdown_connections 2024-05-03T20:26:25.266 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.677+0000 7f08485e3700 1 --2- 172.21.15.181:0/3313776961 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f082803e580 0x7f0828040a40 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:25.266 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.677+0000 7f08485e3700 1 --2- 172.21.15.181:0/3313776961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0840107860 0x7f084019efa0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:25.266 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.677+0000 7f08485e3700 1 -- 172.21.15.181:0/3313776961 >> 172.21.15.181:0/3313776961 conn(0x7f0840103440 msgr2=0x7f0840195c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:25.266 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.677+0000 7f08485e3700 1 -- 172.21.15.181:0/3313776961 shutdown_connections 2024-05-03T20:26:25.267 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:24.677+0000 7f08485e3700 1 -- 172.21.15.181:0/3313776961 wait complete. 2024-05-03T20:26:25.267 INFO:teuthology.orchestra.run.smithi181.stdout:Enabling the dashboard module... 2024-05-03T20:26:25.883 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:25 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3313776961' entity='client.admin' 2024-05-03T20:26:25.884 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:25 smithi181 ceph-mon[29801]: from='mgr.14120 172.21.15.181:0/2717494325' entity='mgr.x' 2024-05-03T20:26:26.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:26 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/161590855' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-03T20:26:26.964 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:26 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: ignoring --setuser ceph since I am not root 2024-05-03T20:26:26.964 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:26 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: ignoring --setgroup ceph since I am not root 2024-05-03T20:26:26.964 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:26 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:26.857+0000 7f4d63387200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T20:26:27.310 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:27 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:27.042+0000 7f4d63387200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T20:26:27.310 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:27 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:27.142+0000 7f4d63387200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T20:26:27.318 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.712+0000 7faf3b641700 1 Processor -- start 2024-05-03T20:26:27.318 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.712+0000 7faf3b641700 1 -- start start 2024-05-03T20:26:27.318 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.712+0000 7faf3b641700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf34108380 0x7faf341087a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:27.318 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.712+0000 7faf3b641700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faf34108d70 con 0x7faf34108380 2024-05-03T20:26:27.318 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.712+0000 7faf393dd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf34108380 0x7faf341087a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:27.318 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.713+0000 7faf393dd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf34108380 0x7faf341087a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35002/0 (socket says 172.21.15.181:35002) 2024-05-03T20:26:27.318 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.713+0000 7faf393dd700 1 -- 172.21.15.181:0/4197630120 learned_addr learned my addr 172.21.15.181:0/4197630120 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:27.318 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.713+0000 7faf393dd700 1 -- 172.21.15.181:0/4197630120 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf341090a0 con 0x7faf34108380 2024-05-03T20:26:27.319 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.713+0000 7faf393dd700 1 --2- 172.21.15.181:0/4197630120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf34108380 0x7faf341087a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faf300098a0 tx=0x7faf30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=803cd839b43e3c34 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:27.319 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.713+0000 7faf2bfff700 1 -- 172.21.15.181:0/4197630120 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf30004160 con 0x7faf34108380 2024-05-03T20:26:27.319 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.713+0000 7faf2bfff700 1 -- 172.21.15.181:0/4197630120 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf30004300 con 0x7faf34108380 2024-05-03T20:26:27.319 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.713+0000 7faf2bfff700 1 -- 172.21.15.181:0/4197630120 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf30011a20 con 0x7faf34108380 2024-05-03T20:26:27.319 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.714+0000 7faf3b641700 1 -- 172.21.15.181:0/4197630120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf34108380 msgr2=0x7faf341087a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:27.319 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.714+0000 7faf3b641700 1 --2- 172.21.15.181:0/4197630120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf34108380 0x7faf341087a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faf300098a0 tx=0x7faf30009bb0 comp rx=0 tx=0).stop 2024-05-03T20:26:27.319 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.714+0000 7faf3b641700 1 -- 172.21.15.181:0/4197630120 shutdown_connections 2024-05-03T20:26:27.319 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.714+0000 7faf3b641700 1 --2- 172.21.15.181:0/4197630120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf34108380 0x7faf341087a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:27.319 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.714+0000 7faf3b641700 1 -- 172.21.15.181:0/4197630120 >> 172.21.15.181:0/4197630120 conn(0x7faf34102be0 msgr2=0x7faf34105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:27.319 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.714+0000 7faf3b641700 1 -- 172.21.15.181:0/4197630120 shutdown_connections 2024-05-03T20:26:27.319 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.714+0000 7faf3b641700 1 -- 172.21.15.181:0/4197630120 wait complete. 2024-05-03T20:26:27.319 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.715+0000 7faf3b641700 1 Processor -- start 2024-05-03T20:26:27.319 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.715+0000 7faf3b641700 1 -- start start 2024-05-03T20:26:27.319 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.715+0000 7faf3b641700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf34108380 0x7faf3419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:27.319 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.715+0000 7faf3b641700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faf34106ee0 con 0x7faf34108380 2024-05-03T20:26:27.320 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.715+0000 7faf393dd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf34108380 0x7faf3419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:27.320 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.715+0000 7faf393dd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf34108380 0x7faf3419ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35006/0 (socket says 172.21.15.181:35006) 2024-05-03T20:26:27.320 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.715+0000 7faf393dd700 1 -- 172.21.15.181:0/161590855 learned_addr learned my addr 172.21.15.181:0/161590855 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:27.320 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.716+0000 7faf393dd700 1 -- 172.21.15.181:0/161590855 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3419b0c0 con 0x7faf34108380 2024-05-03T20:26:27.320 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.716+0000 7faf393dd700 1 --2- 172.21.15.181:0/161590855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf34108380 0x7faf3419ab80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faf30004660 tx=0x7faf30011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:27.320 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.716+0000 7faf2a7fc700 1 -- 172.21.15.181:0/161590855 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf30003aa0 con 0x7faf34108380 2024-05-03T20:26:27.320 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.716+0000 7faf3b641700 1 -- 172.21.15.181:0/161590855 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf3419b350 con 0x7faf34108380 2024-05-03T20:26:27.320 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.716+0000 7faf2a7fc700 1 -- 172.21.15.181:0/161590855 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf3001b8f0 con 0x7faf34108380 2024-05-03T20:26:27.320 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.716+0000 7faf2a7fc700 1 -- 172.21.15.181:0/161590855 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf30023900 con 0x7faf34108380 2024-05-03T20:26:27.320 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.716+0000 7faf3b641700 1 -- 172.21.15.181:0/161590855 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf3419e040 con 0x7faf34108380 2024-05-03T20:26:27.320 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.717+0000 7faf2a7fc700 1 -- 172.21.15.181:0/161590855 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7faf30023b80 con 0x7faf34108380 2024-05-03T20:26:27.320 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.717+0000 7faf3b641700 1 -- 172.21.15.181:0/161590855 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf34064b00 con 0x7faf34108380 2024-05-03T20:26:27.320 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.717+0000 7faf2a7fc700 1 --2- 172.21.15.181:0/161590855 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7faf20040790 0x7faf20042c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:27.320 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.717+0000 7faf2a7fc700 1 -- 172.21.15.181:0/161590855 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faf30054e10 con 0x7faf34108380 2024-05-03T20:26:27.321 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.717+0000 7faf38bdc700 1 --2- 172.21.15.181:0/161590855 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7faf20040790 0x7faf20042c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:27.321 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.718+0000 7faf38bdc700 1 --2- 172.21.15.181:0/161590855 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7faf20040790 0x7faf20042c50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faf24006fd0 tx=0x7faf24006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:27.321 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:25.725+0000 7faf2a7fc700 1 -- 172.21.15.181:0/161590855 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7faf3001f030 con 0x7faf34108380 2024-05-03T20:26:27.321 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:26.013+0000 7faf3b641700 1 -- 172.21.15.181:0/161590855 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7faf341940e0 con 0x7faf34108380 2024-05-03T20:26:27.321 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:26.677+0000 7faf2a7fc700 1 -- 172.21.15.181:0/161590855 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7faf341940e0 con 0x7faf34108380 2024-05-03T20:26:27.321 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:26.678+0000 7faf2a7fc700 1 -- 172.21.15.181:0/161590855 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7faf3001baa0 con 0x7faf34108380 2024-05-03T20:26:27.321 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:26.680+0000 7faf3b641700 1 -- 172.21.15.181:0/161590855 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7faf20040790 msgr2=0x7faf20042c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:27.321 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:26.680+0000 7faf3b641700 1 --2- 172.21.15.181:0/161590855 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7faf20040790 0x7faf20042c50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faf24006fd0 tx=0x7faf24006e40 comp rx=0 tx=0).stop 2024-05-03T20:26:27.321 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:26.680+0000 7faf3b641700 1 -- 172.21.15.181:0/161590855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf34108380 msgr2=0x7faf3419ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:27.321 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:26.680+0000 7faf3b641700 1 --2- 172.21.15.181:0/161590855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf34108380 0x7faf3419ab80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faf30004660 tx=0x7faf30011aa0 comp rx=0 tx=0).stop 2024-05-03T20:26:27.321 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:26.680+0000 7faf3b641700 1 -- 172.21.15.181:0/161590855 shutdown_connections 2024-05-03T20:26:27.321 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:26.680+0000 7faf3b641700 1 --2- 172.21.15.181:0/161590855 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7faf20040790 0x7faf20042c50 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:27.321 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:26.680+0000 7faf3b641700 1 --2- 172.21.15.181:0/161590855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf34108380 0x7faf3419ab80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:27.321 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:26.680+0000 7faf3b641700 1 -- 172.21.15.181:0/161590855 >> 172.21.15.181:0/161590855 conn(0x7faf34102be0 msgr2=0x7faf341911c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:27.322 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:26.680+0000 7faf3b641700 1 -- 172.21.15.181:0/161590855 shutdown_connections 2024-05-03T20:26:27.322 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:26.680+0000 7faf3b641700 1 -- 172.21.15.181:0/161590855 wait complete. 2024-05-03T20:26:27.629 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:27 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:27.455+0000 7f4d63387200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T20:26:27.630 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:27 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:27.544+0000 7f4d63387200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T20:26:27.952 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:27 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/161590855' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-03T20:26:27.952 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:27 smithi181 ceph-mon[29801]: mgrmap e9: x(active, since 20s) 2024-05-03T20:26:28.727 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout { 2024-05-03T20:26:28.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-03T20:26:28.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-03T20:26:28.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-03T20:26:28.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-03T20:26:28.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout } 2024-05-03T20:26:28.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.746+0000 7f1855f5c700 1 Processor -- start 2024-05-03T20:26:28.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.746+0000 7f1855f5c700 1 -- start start 2024-05-03T20:26:28.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.746+0000 7f1855f5c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1850107250 0x7f1850107670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:28.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.747+0000 7f1855f5c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1850107c40 con 0x7f1850107250 2024-05-03T20:26:28.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.747+0000 7f184f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1850107250 0x7f1850107670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:28.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.747+0000 7f184f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1850107250 0x7f1850107670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35022/0 (socket says 172.21.15.181:35022) 2024-05-03T20:26:28.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.747+0000 7f184f7fe700 1 -- 172.21.15.181:0/1519904061 learned_addr learned my addr 172.21.15.181:0/1519904061 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:28.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.747+0000 7f184f7fe700 1 -- 172.21.15.181:0/1519904061 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185010a1e0 con 0x7f1850107250 2024-05-03T20:26:28.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.747+0000 7f184f7fe700 1 --2- 172.21.15.181:0/1519904061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1850107250 0x7f1850107670 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f18400098a0 tx=0x7f1840009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2877de8d0a3476 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:28.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.747+0000 7f184e7fc700 1 -- 172.21.15.181:0/1519904061 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1840004160 con 0x7f1850107250 2024-05-03T20:26:28.729 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.747+0000 7f184e7fc700 1 -- 172.21.15.181:0/1519904061 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1840004300 con 0x7f1850107250 2024-05-03T20:26:28.729 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.747+0000 7f184e7fc700 1 -- 172.21.15.181:0/1519904061 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1840011a20 con 0x7f1850107250 2024-05-03T20:26:28.729 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.748+0000 7f1855f5c700 1 -- 172.21.15.181:0/1519904061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1850107250 msgr2=0x7f1850107670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:28.729 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.748+0000 7f1855f5c700 1 --2- 172.21.15.181:0/1519904061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1850107250 0x7f1850107670 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f18400098a0 tx=0x7f1840009bb0 comp rx=0 tx=0).stop 2024-05-03T20:26:28.729 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.748+0000 7f1855f5c700 1 -- 172.21.15.181:0/1519904061 shutdown_connections 2024-05-03T20:26:28.729 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.748+0000 7f1855f5c700 1 --2- 172.21.15.181:0/1519904061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1850107250 0x7f1850107670 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:28.729 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.748+0000 7f1855f5c700 1 -- 172.21.15.181:0/1519904061 >> 172.21.15.181:0/1519904061 conn(0x7f1850102c00 msgr2=0x7f1850105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:28.729 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.748+0000 7f1855f5c700 1 -- 172.21.15.181:0/1519904061 shutdown_connections 2024-05-03T20:26:28.729 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.748+0000 7f1855f5c700 1 -- 172.21.15.181:0/1519904061 wait complete. 2024-05-03T20:26:28.729 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.749+0000 7f1855f5c700 1 Processor -- start 2024-05-03T20:26:28.729 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.749+0000 7f1855f5c700 1 -- start start 2024-05-03T20:26:28.729 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.750+0000 7f1855f5c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1850107250 0x7f185019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:28.729 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.750+0000 7f1855f5c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f185010d590 con 0x7f1850107250 2024-05-03T20:26:28.729 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.750+0000 7f184f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1850107250 0x7f185019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:28.730 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.750+0000 7f184f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1850107250 0x7f185019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35030/0 (socket says 172.21.15.181:35030) 2024-05-03T20:26:28.730 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.750+0000 7f184f7fe700 1 -- 172.21.15.181:0/871418578 learned_addr learned my addr 172.21.15.181:0/871418578 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:28.730 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.750+0000 7f184f7fe700 1 -- 172.21.15.181:0/871418578 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185019d2e0 con 0x7f1850107250 2024-05-03T20:26:28.730 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.750+0000 7f184f7fe700 1 --2- 172.21.15.181:0/871418578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1850107250 0x7f185019cda0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f18400119f0 tx=0x7f1840011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:28.730 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.750+0000 7f184cff9700 1 -- 172.21.15.181:0/871418578 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18400039e0 con 0x7f1850107250 2024-05-03T20:26:28.730 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.750+0000 7f184cff9700 1 -- 172.21.15.181:0/871418578 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f184001b8f0 con 0x7f1850107250 2024-05-03T20:26:28.730 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.750+0000 7f1855f5c700 1 -- 172.21.15.181:0/871418578 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f185019d570 con 0x7f1850107250 2024-05-03T20:26:28.730 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.750+0000 7f184cff9700 1 -- 172.21.15.181:0/871418578 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18400237a0 con 0x7f1850107250 2024-05-03T20:26:28.730 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.750+0000 7f1855f5c700 1 -- 172.21.15.181:0/871418578 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18501a0260 con 0x7f1850107250 2024-05-03T20:26:28.730 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.751+0000 7f1855f5c700 1 -- 172.21.15.181:0/871418578 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f185010d710 con 0x7f1850107250 2024-05-03T20:26:28.730 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.751+0000 7f184cff9700 1 -- 172.21.15.181:0/871418578 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f184002c3f0 con 0x7f1850107250 2024-05-03T20:26:28.730 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.752+0000 7f184cff9700 1 --2- 172.21.15.181:0/871418578 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f183803e590 0x7f1838040a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:28.730 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.752+0000 7f184cff9700 1 -- 172.21.15.181:0/871418578 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1840054000 con 0x7f1850107250 2024-05-03T20:26:28.731 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.752+0000 7f184effd700 1 -- 172.21.15.181:0/871418578 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f183803e590 msgr2=0x7f1838040a50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6800/2389231557 2024-05-03T20:26:28.731 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.752+0000 7f184effd700 1 --2- 172.21.15.181:0/871418578 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f183803e590 0x7f1838040a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T20:26:28.731 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.757+0000 7f184cff9700 1 -- 172.21.15.181:0/871418578 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f184001f030 con 0x7f1850107250 2024-05-03T20:26:28.731 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.953+0000 7f184effd700 1 -- 172.21.15.181:0/871418578 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f183803e590 msgr2=0x7f1838040a50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6800/2389231557 2024-05-03T20:26:28.731 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:27.953+0000 7f184effd700 1 --2- 172.21.15.181:0/871418578 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f183803e590 0x7f1838040a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-03T20:26:28.731 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:28.058+0000 7f1855f5c700 1 -- 172.21.15.181:0/871418578 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f18501a0430 con 0x7f1850107250 2024-05-03T20:26:28.731 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:28.058+0000 7f184cff9700 1 -- 172.21.15.181:0/871418578 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f18501a0430 con 0x7f1850107250 2024-05-03T20:26:28.731 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:28.061+0000 7f1855f5c700 1 -- 172.21.15.181:0/871418578 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f183803e590 msgr2=0x7f1838040a50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T20:26:28.731 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:28.061+0000 7f1855f5c700 1 --2- 172.21.15.181:0/871418578 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f183803e590 0x7f1838040a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:28.731 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:28.061+0000 7f1855f5c700 1 -- 172.21.15.181:0/871418578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1850107250 msgr2=0x7f185019cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:28.731 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:28.061+0000 7f1855f5c700 1 --2- 172.21.15.181:0/871418578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1850107250 0x7f185019cda0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f18400119f0 tx=0x7f1840011ad0 comp rx=0 tx=0).stop 2024-05-03T20:26:28.731 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:28.061+0000 7f1855f5c700 1 -- 172.21.15.181:0/871418578 shutdown_connections 2024-05-03T20:26:28.731 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:28.061+0000 7f1855f5c700 1 --2- 172.21.15.181:0/871418578 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f183803e590 0x7f1838040a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:28.732 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:28.061+0000 7f1855f5c700 1 --2- 172.21.15.181:0/871418578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1850107250 0x7f185019cda0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:28.732 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:28.061+0000 7f1855f5c700 1 -- 172.21.15.181:0/871418578 >> 172.21.15.181:0/871418578 conn(0x7f1850102c00 msgr2=0x7f1850103620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:28.732 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:28.061+0000 7f1855f5c700 1 -- 172.21.15.181:0/871418578 shutdown_connections 2024-05-03T20:26:28.732 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:28.061+0000 7f1855f5c700 1 -- 172.21.15.181:0/871418578 wait complete. 2024-05-03T20:26:28.732 INFO:teuthology.orchestra.run.smithi181.stdout:Waiting for the mgr to restart... 2024-05-03T20:26:28.732 INFO:teuthology.orchestra.run.smithi181.stdout:Waiting for mgr epoch 9... 2024-05-03T20:26:28.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:28 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/871418578' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-03T20:26:28.963 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:28 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:28.820+0000 7f4d63387200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T20:26:29.329 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:29 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:28.999+0000 7f4d63387200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T20:26:29.329 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:29 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:29.179+0000 7f4d63387200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T20:26:29.713 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:29 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:29.472+0000 7f4d63387200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T20:26:29.713 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:29 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:29.562+0000 7f4d63387200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T20:26:30.213 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:29 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:29.764+0000 7f4d63387200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T20:26:30.213 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:29 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:29.953+0000 7f4d63387200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T20:26:30.963 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:30 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:30.547+0000 7f4d63387200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T20:26:30.963 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:30 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:30.764+0000 7f4d63387200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T20:26:31.463 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:31 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:31.180+0000 7f4d63387200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T20:26:31.463 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:31 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:31.274+0000 7f4d63387200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T20:26:31.963 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:31 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:31.641+0000 7f4d63387200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T20:26:32.320 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:31 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:31.991+0000 7f4d63387200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T20:26:32.320 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:32 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:32.082+0000 7f4d63387200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T20:26:32.713 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:32 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:32.320+0000 7f4d63387200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T20:26:32.713 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:32 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:32.451+0000 7f4d63387200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T20:26:33.213 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:32 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:32.755+0000 7f4d63387200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T20:26:33.213 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:32 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:32.869+0000 7f4d63387200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T20:26:33.213 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:32 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:32.959+0000 7f4d63387200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T20:26:33.817 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:33 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:33.549+0000 7f4d63387200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T20:26:33.818 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:26:33 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:26:33.765+0000 7f4d63387200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T20:26:34.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:33 smithi181 ceph-mon[29801]: Active manager daemon x restarted 2024-05-03T20:26:34.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:33 smithi181 ceph-mon[29801]: Activating manager daemon x 2024-05-03T20:26:34.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:33 smithi181 ceph-mon[29801]: osdmap e3: 0 total, 0 up, 0 in 2024-05-03T20:26:34.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:33 smithi181 ceph-mon[29801]: mgrmap e10: x(active, starting, since 0.00414464s) 2024-05-03T20:26:34.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:33 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T20:26:34.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:33 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-03T20:26:34.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:33 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-03T20:26:34.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:33 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-03T20:26:34.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:33 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-03T20:26:34.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:33 smithi181 ceph-mon[29801]: Manager daemon x is now available 2024-05-03T20:26:34.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:33 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:26:34.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:33 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-03T20:26:34.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:34 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-03T20:26:34.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:34 smithi181 ceph-mon[29801]: [03/May/2024:20:26:33] ENGINE Bus STARTING 2024-05-03T20:26:34.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:34 smithi181 ceph-mon[29801]: [03/May/2024:20:26:33] ENGINE Serving on http://172.21.15.181:8765 2024-05-03T20:26:34.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:34 smithi181 ceph-mon[29801]: [03/May/2024:20:26:34] ENGINE Serving on https://172.21.15.181:7150 2024-05-03T20:26:34.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:34 smithi181 ceph-mon[29801]: [03/May/2024:20:26:34] ENGINE Bus STARTED 2024-05-03T20:26:34.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:34 smithi181 ceph-mon[29801]: mgrmap e11: x(active, since 1.00648s) 2024-05-03T20:26:35.585 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout { 2024-05-03T20:26:35.586 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-03T20:26:35.586 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-03T20:26:35.586 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout } 2024-05-03T20:26:35.586 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.134+0000 7f501f455700 1 Processor -- start 2024-05-03T20:26:35.586 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.134+0000 7f501f455700 1 -- start start 2024-05-03T20:26:35.586 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.135+0000 7f501f455700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5018107e80 0x7f50181082a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:35.586 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.135+0000 7f501f455700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f50181087e0 con 0x7f5018107e80 2024-05-03T20:26:35.586 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.135+0000 7f501d1f1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5018107e80 0x7f50181082a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:35.586 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.135+0000 7f501d1f1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5018107e80 0x7f50181082a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35034/0 (socket says 172.21.15.181:35034) 2024-05-03T20:26:35.586 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.135+0000 7f501d1f1700 1 -- 172.21.15.181:0/1711048526 learned_addr learned my addr 172.21.15.181:0/1711048526 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:35.586 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.135+0000 7f501d1f1700 1 -- 172.21.15.181:0/1711048526 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5018108fe0 con 0x7f5018107e80 2024-05-03T20:26:35.586 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.135+0000 7f501d1f1700 1 --2- 172.21.15.181:0/1711048526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5018107e80 0x7f50181082a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f50140098a0 tx=0x7f5014009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57e251de3789090 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:35.587 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.136+0000 7f500ffff700 1 -- 172.21.15.181:0/1711048526 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5014004160 con 0x7f5018107e80 2024-05-03T20:26:35.587 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.136+0000 7f500ffff700 1 -- 172.21.15.181:0/1711048526 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5014004300 con 0x7f5018107e80 2024-05-03T20:26:35.587 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.136+0000 7f500ffff700 1 -- 172.21.15.181:0/1711048526 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5014011a20 con 0x7f5018107e80 2024-05-03T20:26:35.587 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.137+0000 7f501f455700 1 -- 172.21.15.181:0/1711048526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5018107e80 msgr2=0x7f50181082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:35.587 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.137+0000 7f501f455700 1 --2- 172.21.15.181:0/1711048526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5018107e80 0x7f50181082a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f50140098a0 tx=0x7f5014009bb0 comp rx=0 tx=0).stop 2024-05-03T20:26:35.587 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.137+0000 7f501f455700 1 -- 172.21.15.181:0/1711048526 shutdown_connections 2024-05-03T20:26:35.587 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.137+0000 7f501f455700 1 --2- 172.21.15.181:0/1711048526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5018107e80 0x7f50181082a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:35.587 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.137+0000 7f501f455700 1 -- 172.21.15.181:0/1711048526 >> 172.21.15.181:0/1711048526 conn(0x7f5018103480 msgr2=0x7f50181058a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:35.587 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.137+0000 7f501f455700 1 -- 172.21.15.181:0/1711048526 shutdown_connections 2024-05-03T20:26:35.587 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.137+0000 7f501f455700 1 -- 172.21.15.181:0/1711048526 wait complete. 2024-05-03T20:26:35.587 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.138+0000 7f501f455700 1 Processor -- start 2024-05-03T20:26:35.587 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.138+0000 7f501f455700 1 -- start start 2024-05-03T20:26:35.587 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.138+0000 7f501f455700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5018107e80 0x7f5018076960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:35.587 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.138+0000 7f501f455700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f501810c420 con 0x7f5018107e80 2024-05-03T20:26:35.587 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.138+0000 7f501d1f1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5018107e80 0x7f5018076960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:35.588 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.138+0000 7f501d1f1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5018107e80 0x7f5018076960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35036/0 (socket says 172.21.15.181:35036) 2024-05-03T20:26:35.588 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.138+0000 7f501d1f1700 1 -- 172.21.15.181:0/1101123796 learned_addr learned my addr 172.21.15.181:0/1101123796 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:35.588 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.139+0000 7f501d1f1700 1 -- 172.21.15.181:0/1101123796 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f501819faa0 con 0x7f5018107e80 2024-05-03T20:26:35.588 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.139+0000 7f501d1f1700 1 --2- 172.21.15.181:0/1101123796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5018107e80 0x7f5018076960 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5014004660 tx=0x7f5014011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:35.588 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.139+0000 7f500e7fc700 1 -- 172.21.15.181:0/1101123796 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5014003aa0 con 0x7f5018107e80 2024-05-03T20:26:35.588 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.139+0000 7f500e7fc700 1 -- 172.21.15.181:0/1101123796 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f501401b8f0 con 0x7f5018107e80 2024-05-03T20:26:35.588 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.139+0000 7f500e7fc700 1 -- 172.21.15.181:0/1101123796 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5014023900 con 0x7f5018107e80 2024-05-03T20:26:35.588 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.140+0000 7f501f455700 1 -- 172.21.15.181:0/1101123796 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f501819fc70 con 0x7f5018107e80 2024-05-03T20:26:35.588 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.140+0000 7f501f455700 1 -- 172.21.15.181:0/1101123796 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50181a0090 con 0x7f5018107e80 2024-05-03T20:26:35.588 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.141+0000 7f500e7fc700 1 -- 172.21.15.181:0/1101123796 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f501401baa0 con 0x7f5018107e80 2024-05-03T20:26:35.588 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.141+0000 7f500e7fc700 1 --2- 172.21.15.181:0/1101123796 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f500403e580 0x7f5004040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:35.588 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.141+0000 7f500e7fc700 1 -- 172.21.15.181:0/1101123796 --> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5004041150 con 0x7f500403e580 2024-05-03T20:26:35.588 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.141+0000 7f501c9f0700 1 -- 172.21.15.181:0/1101123796 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f500403e580 msgr2=0x7f5004040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6800/2389231557 2024-05-03T20:26:35.588 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.141+0000 7f500e7fc700 1 -- 172.21.15.181:0/1101123796 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f50140541d0 con 0x7f5018107e80 2024-05-03T20:26:35.589 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.141+0000 7f501c9f0700 1 --2- 172.21.15.181:0/1101123796 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f500403e580 0x7f5004040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T20:26:35.589 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.342+0000 7f501c9f0700 1 -- 172.21.15.181:0/1101123796 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f500403e580 msgr2=0x7f5004040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6800/2389231557 2024-05-03T20:26:35.589 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.342+0000 7f501c9f0700 1 --2- 172.21.15.181:0/1101123796 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f500403e580 0x7f5004040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-03T20:26:35.589 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.743+0000 7f501c9f0700 1 -- 172.21.15.181:0/1101123796 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f500403e580 msgr2=0x7f5004040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6800/2389231557 2024-05-03T20:26:35.589 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:29.743+0000 7f501c9f0700 1 --2- 172.21.15.181:0/1101123796 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f500403e580 0x7f5004040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-03T20:26:35.589 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:30.544+0000 7f501c9f0700 1 -- 172.21.15.181:0/1101123796 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f500403e580 msgr2=0x7f5004040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6800/2389231557 2024-05-03T20:26:35.589 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:30.544+0000 7f501c9f0700 1 --2- 172.21.15.181:0/1101123796 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f500403e580 0x7f5004040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-03T20:26:35.589 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:32.145+0000 7f501c9f0700 1 -- 172.21.15.181:0/1101123796 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f500403e580 msgr2=0x7f5004040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6800/2389231557 2024-05-03T20:26:35.589 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:32.145+0000 7f501c9f0700 1 --2- 172.21.15.181:0/1101123796 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f500403e580 0x7f5004040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-03T20:26:35.589 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:33.769+0000 7f500e7fc700 1 -- 172.21.15.181:0/1101123796 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7f5014023b30 con 0x7f5018107e80 2024-05-03T20:26:35.589 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:33.769+0000 7f500e7fc700 1 -- 172.21.15.181:0/1101123796 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f500403e580 msgr2=0x7f5004040a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T20:26:35.589 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:33.769+0000 7f500e7fc700 1 --2- 172.21.15.181:0/1101123796 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f500403e580 0x7f5004040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:35.589 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:34.770+0000 7f500e7fc700 1 -- 172.21.15.181:0/1101123796 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f5014055d40 con 0x7f5018107e80 2024-05-03T20:26:35.589 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:34.771+0000 7f500e7fc700 1 --2- 172.21.15.181:0/1101123796 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5004041fb0 0x7f50040443a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:35.589 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:34.771+0000 7f500e7fc700 1 -- 172.21.15.181:0/1101123796 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5014017ec0 con 0x7f5004041fb0 2024-05-03T20:26:35.590 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:34.771+0000 7f501c9f0700 1 --2- 172.21.15.181:0/1101123796 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5004041fb0 0x7f50040443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:35.590 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:34.771+0000 7f501c9f0700 1 --2- 172.21.15.181:0/1101123796 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5004041fb0 0x7f50040443a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5008003a10 tx=0x7f50080092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:35.590 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:34.772+0000 7f500e7fc700 1 -- 172.21.15.181:0/1101123796 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f5014017ec0 con 0x7f5004041fb0 2024-05-03T20:26:35.590 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:34.781+0000 7f501f455700 1 -- 172.21.15.181:0/1101123796 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f5018064b00 con 0x7f5004041fb0 2024-05-03T20:26:35.590 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:34.781+0000 7f500e7fc700 1 -- 172.21.15.181:0/1101123796 <== mgr.14152 v2:172.21.15.181:6800/1919423525 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f5018064b00 con 0x7f5004041fb0 2024-05-03T20:26:35.590 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:34.781+0000 7f501f455700 1 -- 172.21.15.181:0/1101123796 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5004041fb0 msgr2=0x7f50040443a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:35.590 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:34.781+0000 7f501f455700 1 --2- 172.21.15.181:0/1101123796 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5004041fb0 0x7f50040443a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5008003a10 tx=0x7f50080092b0 comp rx=0 tx=0).stop 2024-05-03T20:26:35.590 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:34.781+0000 7f501f455700 1 -- 172.21.15.181:0/1101123796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5018107e80 msgr2=0x7f5018076960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:35.590 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:34.781+0000 7f501f455700 1 --2- 172.21.15.181:0/1101123796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5018107e80 0x7f5018076960 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5014004660 tx=0x7f5014011aa0 comp rx=0 tx=0).stop 2024-05-03T20:26:35.590 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:34.781+0000 7f501f455700 1 -- 172.21.15.181:0/1101123796 shutdown_connections 2024-05-03T20:26:35.590 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:34.781+0000 7f501f455700 1 --2- 172.21.15.181:0/1101123796 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5004041fb0 0x7f50040443a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:35.590 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:34.781+0000 7f501f455700 1 --2- 172.21.15.181:0/1101123796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5018107e80 0x7f5018076960 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:35.590 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:34.781+0000 7f501f455700 1 --2- 172.21.15.181:0/1101123796 >> [v2:172.21.15.181:6800/2389231557,v1:172.21.15.181:6801/2389231557] conn(0x7f500403e580 0x7f5004040a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:35.590 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:34.781+0000 7f501f455700 1 -- 172.21.15.181:0/1101123796 >> 172.21.15.181:0/1101123796 conn(0x7f5018103480 msgr2=0x7f5018103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:35.591 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:34.782+0000 7f501f455700 1 -- 172.21.15.181:0/1101123796 shutdown_connections 2024-05-03T20:26:35.591 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:34.782+0000 7f501f455700 1 -- 172.21.15.181:0/1101123796 wait complete. 2024-05-03T20:26:35.591 INFO:teuthology.orchestra.run.smithi181.stdout:mgr epoch 9 is available 2024-05-03T20:26:35.591 INFO:teuthology.orchestra.run.smithi181.stdout:Generating a dashboard self-signed certificate... 2024-05-03T20:26:35.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:35 smithi181 ceph-mon[29801]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-03T20:26:35.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:35 smithi181 ceph-mon[29801]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-03T20:26:37.041 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:36 smithi181 ceph-mon[29801]: mgrmap e12: x(active, since 2s) 2024-05-03T20:26:37.041 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:26:37.041 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:36 smithi181 ceph-mon[29801]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:26:37.041 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:26:37.041 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:26:37.049 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-03T20:26:37.049 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.029+0000 7f86ca825700 1 Processor -- start 2024-05-03T20:26:37.049 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.029+0000 7f86ca825700 1 -- start start 2024-05-03T20:26:37.049 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.029+0000 7f86ca825700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c407ee60 0x7f86c407d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:37.049 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.029+0000 7f86ca825700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f86c407d890 con 0x7f86c407ee60 2024-05-03T20:26:37.049 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.029+0000 7f86c3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c407ee60 0x7f86c407d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:37.049 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.029+0000 7f86c3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c407ee60 0x7f86c407d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60170/0 (socket says 172.21.15.181:60170) 2024-05-03T20:26:37.049 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.029+0000 7f86c3fff700 1 -- 172.21.15.181:0/3324400453 learned_addr learned my addr 172.21.15.181:0/3324400453 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:37.049 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.029+0000 7f86c3fff700 1 -- 172.21.15.181:0/3324400453 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c407da10 con 0x7f86c407ee60 2024-05-03T20:26:37.049 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.029+0000 7f86c3fff700 1 --2- 172.21.15.181:0/3324400453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c407ee60 0x7f86c407d2c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f86b40098a0 tx=0x7f86b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792db6426b7c7b44 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:37.049 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.030+0000 7f86c2ffd700 1 -- 172.21.15.181:0/3324400453 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86b4004160 con 0x7f86c407ee60 2024-05-03T20:26:37.049 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.030+0000 7f86c2ffd700 1 -- 172.21.15.181:0/3324400453 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86b4004300 con 0x7f86c407ee60 2024-05-03T20:26:37.049 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.030+0000 7f86c2ffd700 1 -- 172.21.15.181:0/3324400453 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86b4011a20 con 0x7f86c407ee60 2024-05-03T20:26:37.050 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.030+0000 7f86ca825700 1 -- 172.21.15.181:0/3324400453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c407ee60 msgr2=0x7f86c407d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:37.050 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.030+0000 7f86ca825700 1 --2- 172.21.15.181:0/3324400453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c407ee60 0x7f86c407d2c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f86b40098a0 tx=0x7f86b4009bb0 comp rx=0 tx=0).stop 2024-05-03T20:26:37.050 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.030+0000 7f86ca825700 1 -- 172.21.15.181:0/3324400453 shutdown_connections 2024-05-03T20:26:37.050 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.030+0000 7f86ca825700 1 --2- 172.21.15.181:0/3324400453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c407ee60 0x7f86c407d2c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:37.050 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.030+0000 7f86ca825700 1 -- 172.21.15.181:0/3324400453 >> 172.21.15.181:0/3324400453 conn(0x7f86c4104550 msgr2=0x7f86c4106970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:37.050 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.030+0000 7f86ca825700 1 -- 172.21.15.181:0/3324400453 shutdown_connections 2024-05-03T20:26:37.050 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.030+0000 7f86ca825700 1 -- 172.21.15.181:0/3324400453 wait complete. 2024-05-03T20:26:37.050 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.031+0000 7f86ca825700 1 Processor -- start 2024-05-03T20:26:37.050 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.031+0000 7f86ca825700 1 -- start start 2024-05-03T20:26:37.050 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.031+0000 7f86ca825700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c407ee60 0x7f86c419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:37.050 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.031+0000 7f86ca825700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f86c410f590 con 0x7f86c407ee60 2024-05-03T20:26:37.050 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.031+0000 7f86c3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c407ee60 0x7f86c419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:37.050 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.031+0000 7f86c3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c407ee60 0x7f86c419eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60186/0 (socket says 172.21.15.181:60186) 2024-05-03T20:26:37.051 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.031+0000 7f86c3fff700 1 -- 172.21.15.181:0/3947775452 learned_addr learned my addr 172.21.15.181:0/3947775452 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:37.051 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.031+0000 7f86c3fff700 1 -- 172.21.15.181:0/3947775452 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c419f3e0 con 0x7f86c407ee60 2024-05-03T20:26:37.051 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.032+0000 7f86c3fff700 1 --2- 172.21.15.181:0/3947775452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c407ee60 0x7f86c419eea0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f86b40116e0 tx=0x7f86b4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:37.051 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.032+0000 7f86c17fa700 1 -- 172.21.15.181:0/3947775452 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86b4003a60 con 0x7f86c407ee60 2024-05-03T20:26:37.051 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.032+0000 7f86c17fa700 1 -- 172.21.15.181:0/3947775452 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86b401b8f0 con 0x7f86c407ee60 2024-05-03T20:26:37.051 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.032+0000 7f86ca825700 1 -- 172.21.15.181:0/3947775452 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c419f670 con 0x7f86c407ee60 2024-05-03T20:26:37.051 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.032+0000 7f86c17fa700 1 -- 172.21.15.181:0/3947775452 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86b4019860 con 0x7f86c407ee60 2024-05-03T20:26:37.051 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.032+0000 7f86ca825700 1 -- 172.21.15.181:0/3947775452 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c41a2360 con 0x7f86c407ee60 2024-05-03T20:26:37.051 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.033+0000 7f86ca825700 1 -- 172.21.15.181:0/3947775452 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86c4064b00 con 0x7f86c407ee60 2024-05-03T20:26:37.051 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.033+0000 7f86c17fa700 1 -- 172.21.15.181:0/3947775452 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f86b4019b60 con 0x7f86c407ee60 2024-05-03T20:26:37.051 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.033+0000 7f86c17fa700 1 --2- 172.21.15.181:0/3947775452 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f86ac03e530 0x7f86ac0409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:37.051 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.033+0000 7f86c17fa700 1 -- 172.21.15.181:0/3947775452 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f86b4053db0 con 0x7f86c407ee60 2024-05-03T20:26:37.051 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.033+0000 7f86c37fe700 1 --2- 172.21.15.181:0/3947775452 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f86ac03e530 0x7f86ac0409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:37.051 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.034+0000 7f86c37fe700 1 --2- 172.21.15.181:0/3947775452 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f86ac03e530 0x7f86ac0409f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f86b8006fd0 tx=0x7f86b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:37.051 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.038+0000 7f86c17fa700 1 -- 172.21.15.181:0/3947775452 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f86b401f030 con 0x7f86c407ee60 2024-05-03T20:26:37.052 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.267+0000 7f86ca825700 1 -- 172.21.15.181:0/3947775452 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f86c4105570 con 0x7f86ac03e530 2024-05-03T20:26:37.052 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.443+0000 7f86c17fa700 1 -- 172.21.15.181:0/3947775452 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f86c4105570 con 0x7f86ac03e530 2024-05-03T20:26:37.052 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.445+0000 7f86ca825700 1 -- 172.21.15.181:0/3947775452 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f86ac03e530 msgr2=0x7f86ac0409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:37.052 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.445+0000 7f86ca825700 1 --2- 172.21.15.181:0/3947775452 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f86ac03e530 0x7f86ac0409f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f86b8006fd0 tx=0x7f86b8006e40 comp rx=0 tx=0).stop 2024-05-03T20:26:37.052 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.445+0000 7f86ca825700 1 -- 172.21.15.181:0/3947775452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c407ee60 msgr2=0x7f86c419eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:37.052 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.445+0000 7f86ca825700 1 --2- 172.21.15.181:0/3947775452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c407ee60 0x7f86c419eea0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f86b40116e0 tx=0x7f86b4011aa0 comp rx=0 tx=0).stop 2024-05-03T20:26:37.052 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.445+0000 7f86ca825700 1 -- 172.21.15.181:0/3947775452 shutdown_connections 2024-05-03T20:26:37.052 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.445+0000 7f86ca825700 1 --2- 172.21.15.181:0/3947775452 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f86ac03e530 0x7f86ac0409f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:37.052 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.445+0000 7f86ca825700 1 --2- 172.21.15.181:0/3947775452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c407ee60 0x7f86c419eea0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:37.052 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.445+0000 7f86ca825700 1 -- 172.21.15.181:0/3947775452 >> 172.21.15.181:0/3947775452 conn(0x7f86c4104550 msgr2=0x7f86c4104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:37.052 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.445+0000 7f86ca825700 1 -- 172.21.15.181:0/3947775452 shutdown_connections 2024-05-03T20:26:37.052 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:36.445+0000 7f86ca825700 1 -- 172.21.15.181:0/3947775452 wait complete. 2024-05-03T20:26:37.052 INFO:teuthology.orchestra.run.smithi181.stdout:Creating initial admin user... 2024-05-03T20:26:38.708 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:38 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:26:38.708 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:38 smithi181 ceph-mon[29801]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:26:38.708 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:38 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:26:38.716 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$stbxsyPy5wRHXCbEBG9pT.dtqlH0TsEfkVS32jKbbW1jNOHa/NxB6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714767997, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-03T20:26:38.716 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.460+0000 7fee4090b700 1 Processor -- start 2024-05-03T20:26:38.716 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.460+0000 7fee4090b700 1 -- start start 2024-05-03T20:26:38.716 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.460+0000 7fee4090b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c108f90 0x7fee3c1093b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:38.716 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.460+0000 7fee4090b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c109980 con 0x7fee3c108f90 2024-05-03T20:26:38.716 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.461+0000 7fee3a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c108f90 0x7fee3c1093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:38.717 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.461+0000 7fee3a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c108f90 0x7fee3c1093b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60194/0 (socket says 172.21.15.181:60194) 2024-05-03T20:26:38.717 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.461+0000 7fee3a59c700 1 -- 172.21.15.181:0/598323220 learned_addr learned my addr 172.21.15.181:0/598323220 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:38.717 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.461+0000 7fee3a59c700 1 -- 172.21.15.181:0/598323220 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3c10a190 con 0x7fee3c108f90 2024-05-03T20:26:38.717 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.461+0000 7fee3a59c700 1 --2- 172.21.15.181:0/598323220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c108f90 0x7fee3c1093b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fee300098a0 tx=0x7fee30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a484c2377dc5407d server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:38.717 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.461+0000 7fee3959a700 1 -- 172.21.15.181:0/598323220 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee30004160 con 0x7fee3c108f90 2024-05-03T20:26:38.717 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.461+0000 7fee3959a700 1 -- 172.21.15.181:0/598323220 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee30004300 con 0x7fee3c108f90 2024-05-03T20:26:38.717 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.461+0000 7fee3959a700 1 -- 172.21.15.181:0/598323220 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee30011a20 con 0x7fee3c108f90 2024-05-03T20:26:38.717 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.462+0000 7fee4090b700 1 -- 172.21.15.181:0/598323220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c108f90 msgr2=0x7fee3c1093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:38.717 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.462+0000 7fee4090b700 1 --2- 172.21.15.181:0/598323220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c108f90 0x7fee3c1093b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fee300098a0 tx=0x7fee30009bb0 comp rx=0 tx=0).stop 2024-05-03T20:26:38.717 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.462+0000 7fee4090b700 1 -- 172.21.15.181:0/598323220 shutdown_connections 2024-05-03T20:26:38.717 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.462+0000 7fee4090b700 1 --2- 172.21.15.181:0/598323220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c108f90 0x7fee3c1093b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:38.717 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.462+0000 7fee4090b700 1 -- 172.21.15.181:0/598323220 >> 172.21.15.181:0/598323220 conn(0x7fee3c104550 msgr2=0x7fee3c106970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:38.717 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.462+0000 7fee4090b700 1 -- 172.21.15.181:0/598323220 shutdown_connections 2024-05-03T20:26:38.717 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.462+0000 7fee4090b700 1 -- 172.21.15.181:0/598323220 wait complete. 2024-05-03T20:26:38.717 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.463+0000 7fee4090b700 1 Processor -- start 2024-05-03T20:26:38.717 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.463+0000 7fee4090b700 1 -- start start 2024-05-03T20:26:38.718 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.463+0000 7fee4090b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c108f90 0x7fee3c07eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:38.718 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.463+0000 7fee4090b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c10d540 con 0x7fee3c108f90 2024-05-03T20:26:38.718 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.463+0000 7fee3a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c108f90 0x7fee3c07eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:38.718 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.463+0000 7fee3a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c108f90 0x7fee3c07eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60208/0 (socket says 172.21.15.181:60208) 2024-05-03T20:26:38.718 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.463+0000 7fee3a59c700 1 -- 172.21.15.181:0/2444940087 learned_addr learned my addr 172.21.15.181:0/2444940087 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:38.718 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.463+0000 7fee3a59c700 1 -- 172.21.15.181:0/2444940087 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3c07f090 con 0x7fee3c108f90 2024-05-03T20:26:38.718 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.463+0000 7fee3a59c700 1 --2- 172.21.15.181:0/2444940087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c108f90 0x7fee3c07eb50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fee300116e0 tx=0x7fee30011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:38.718 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.463+0000 7fee2b7fe700 1 -- 172.21.15.181:0/2444940087 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee300039e0 con 0x7fee3c108f90 2024-05-03T20:26:38.718 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.464+0000 7fee2b7fe700 1 -- 172.21.15.181:0/2444940087 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee3001b8f0 con 0x7fee3c108f90 2024-05-03T20:26:38.718 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.464+0000 7fee4090b700 1 -- 172.21.15.181:0/2444940087 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee3c07d280 con 0x7fee3c108f90 2024-05-03T20:26:38.718 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.464+0000 7fee2b7fe700 1 -- 172.21.15.181:0/2444940087 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee30023760 con 0x7fee3c108f90 2024-05-03T20:26:38.718 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.464+0000 7fee4090b700 1 -- 172.21.15.181:0/2444940087 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee3c07d7c0 con 0x7fee3c108f90 2024-05-03T20:26:38.718 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.464+0000 7fee2b7fe700 1 -- 172.21.15.181:0/2444940087 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fee300239d0 con 0x7fee3c108f90 2024-05-03T20:26:38.718 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.465+0000 7fee2b7fe700 1 --2- 172.21.15.181:0/2444940087 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fee2403e530 0x7fee240409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:38.719 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.465+0000 7fee2b7fe700 1 -- 172.21.15.181:0/2444940087 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fee30053fd0 con 0x7fee3c108f90 2024-05-03T20:26:38.719 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.465+0000 7fee39d9b700 1 --2- 172.21.15.181:0/2444940087 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fee2403e530 0x7fee240409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:38.719 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.465+0000 7fee4090b700 1 -- 172.21.15.181:0/2444940087 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee3c10d6c0 con 0x7fee3c108f90 2024-05-03T20:26:38.719 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.465+0000 7fee39d9b700 1 --2- 172.21.15.181:0/2444940087 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fee2403e530 0x7fee240409f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fee2c006fd0 tx=0x7fee2c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:38.719 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.470+0000 7fee2b7fe700 1 -- 172.21.15.181:0/2444940087 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fee3001f030 con 0x7fee3c108f90 2024-05-03T20:26:38.719 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.715+0000 7fee4090b700 1 -- 172.21.15.181:0/2444940087 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fee3c064b00 con 0x7fee2403e530 2024-05-03T20:26:38.719 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.941+0000 7fee2b7fe700 1 -- 172.21.15.181:0/2444940087 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fee3c064b00 con 0x7fee2403e530 2024-05-03T20:26:38.719 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.944+0000 7fee4090b700 1 -- 172.21.15.181:0/2444940087 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fee2403e530 msgr2=0x7fee240409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:38.719 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.944+0000 7fee4090b700 1 --2- 172.21.15.181:0/2444940087 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fee2403e530 0x7fee240409f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fee2c006fd0 tx=0x7fee2c006e40 comp rx=0 tx=0).stop 2024-05-03T20:26:38.719 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.944+0000 7fee4090b700 1 -- 172.21.15.181:0/2444940087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c108f90 msgr2=0x7fee3c07eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:38.719 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.944+0000 7fee4090b700 1 --2- 172.21.15.181:0/2444940087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c108f90 0x7fee3c07eb50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fee300116e0 tx=0x7fee30011aa0 comp rx=0 tx=0).stop 2024-05-03T20:26:38.719 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.944+0000 7fee4090b700 1 -- 172.21.15.181:0/2444940087 shutdown_connections 2024-05-03T20:26:38.719 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.944+0000 7fee4090b700 1 --2- 172.21.15.181:0/2444940087 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fee2403e530 0x7fee240409f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:38.719 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.944+0000 7fee4090b700 1 --2- 172.21.15.181:0/2444940087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c108f90 0x7fee3c07eb50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:38.719 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.944+0000 7fee4090b700 1 -- 172.21.15.181:0/2444940087 >> 172.21.15.181:0/2444940087 conn(0x7fee3c104550 msgr2=0x7fee3c104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:38.720 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.944+0000 7fee4090b700 1 -- 172.21.15.181:0/2444940087 shutdown_connections 2024-05-03T20:26:38.720 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:37.944+0000 7fee4090b700 1 -- 172.21.15.181:0/2444940087 wait complete. 2024-05-03T20:26:38.720 INFO:teuthology.orchestra.run.smithi181.stdout:Fetching dashboard port number... 2024-05-03T20:26:39.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:39 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/662069796' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-03T20:26:40.462 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stdout 8443 2024-05-03T20:26:40.462 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.154+0000 7f675333b700 1 Processor -- start 2024-05-03T20:26:40.462 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.155+0000 7f675333b700 1 -- start start 2024-05-03T20:26:40.462 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.155+0000 7f675333b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f674c109a10 0x7f674c10be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:40.462 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.155+0000 7f675333b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f674c07ce90 con 0x7f674c109a10 2024-05-03T20:26:40.462 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.155+0000 7f67510d7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f674c109a10 0x7f674c10be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:40.462 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.155+0000 7f67510d7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f674c109a10 0x7f674c10be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60224/0 (socket says 172.21.15.181:60224) 2024-05-03T20:26:40.462 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.155+0000 7f67510d7700 1 -- 172.21.15.181:0/3896636597 learned_addr learned my addr 172.21.15.181:0/3896636597 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:40.462 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.155+0000 7f67510d7700 1 -- 172.21.15.181:0/3896636597 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f674c10c340 con 0x7f674c109a10 2024-05-03T20:26:40.462 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.155+0000 7f67510d7700 1 --2- 172.21.15.181:0/3896636597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f674c109a10 0x7f674c10be00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f67480098a0 tx=0x7f6748009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2095fab0aaa36d3e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:40.462 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.156+0000 7f6743fff700 1 -- 172.21.15.181:0/3896636597 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6748004160 con 0x7f674c109a10 2024-05-03T20:26:40.462 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.156+0000 7f6743fff700 1 -- 172.21.15.181:0/3896636597 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6748004300 con 0x7f674c109a10 2024-05-03T20:26:40.463 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.156+0000 7f6743fff700 1 -- 172.21.15.181:0/3896636597 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6748011a20 con 0x7f674c109a10 2024-05-03T20:26:40.463 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.156+0000 7f675333b700 1 -- 172.21.15.181:0/3896636597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f674c109a10 msgr2=0x7f674c10be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:40.463 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.156+0000 7f675333b700 1 --2- 172.21.15.181:0/3896636597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f674c109a10 0x7f674c10be00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f67480098a0 tx=0x7f6748009bb0 comp rx=0 tx=0).stop 2024-05-03T20:26:40.463 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.156+0000 7f675333b700 1 -- 172.21.15.181:0/3896636597 shutdown_connections 2024-05-03T20:26:40.463 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.156+0000 7f675333b700 1 --2- 172.21.15.181:0/3896636597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f674c109a10 0x7f674c10be00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:40.463 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.156+0000 7f675333b700 1 -- 172.21.15.181:0/3896636597 >> 172.21.15.181:0/3896636597 conn(0x7f674c103460 msgr2=0x7f674c105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:40.463 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.156+0000 7f675333b700 1 -- 172.21.15.181:0/3896636597 shutdown_connections 2024-05-03T20:26:40.463 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.156+0000 7f675333b700 1 -- 172.21.15.181:0/3896636597 wait complete. 2024-05-03T20:26:40.463 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.157+0000 7f675333b700 1 Processor -- start 2024-05-03T20:26:40.463 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.157+0000 7f675333b700 1 -- start start 2024-05-03T20:26:40.463 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.157+0000 7f675333b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f674c109a10 0x7f674c109290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:40.463 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.157+0000 7f675333b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f674c10d350 con 0x7f674c109a10 2024-05-03T20:26:40.463 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.157+0000 7f67510d7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f674c109a10 0x7f674c109290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:40.463 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.157+0000 7f67510d7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f674c109a10 0x7f674c109290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60236/0 (socket says 172.21.15.181:60236) 2024-05-03T20:26:40.464 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.157+0000 7f67510d7700 1 -- 172.21.15.181:0/662069796 learned_addr learned my addr 172.21.15.181:0/662069796 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:40.464 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.157+0000 7f67510d7700 1 -- 172.21.15.181:0/662069796 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f674c1097d0 con 0x7f674c109a10 2024-05-03T20:26:40.464 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.158+0000 7f67510d7700 1 --2- 172.21.15.181:0/662069796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f674c109a10 0x7f674c109290 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f67480116e0 tx=0x7f6748011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:40.464 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.158+0000 7f67427fc700 1 -- 172.21.15.181:0/662069796 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6748003a60 con 0x7f674c109a10 2024-05-03T20:26:40.464 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.158+0000 7f67427fc700 1 -- 172.21.15.181:0/662069796 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f674801b8f0 con 0x7f674c109a10 2024-05-03T20:26:40.464 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.158+0000 7f67427fc700 1 -- 172.21.15.181:0/662069796 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6748019860 con 0x7f674c109a10 2024-05-03T20:26:40.464 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.158+0000 7f675333b700 1 -- 172.21.15.181:0/662069796 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f674c1079c0 con 0x7f674c109a10 2024-05-03T20:26:40.464 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.158+0000 7f675333b700 1 -- 172.21.15.181:0/662069796 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f674c107f00 con 0x7f674c109a10 2024-05-03T20:26:40.464 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.159+0000 7f67427fc700 1 -- 172.21.15.181:0/662069796 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f674801b440 con 0x7f674c109a10 2024-05-03T20:26:40.464 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.159+0000 7f675333b700 1 -- 172.21.15.181:0/662069796 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f674c064b00 con 0x7f674c109a10 2024-05-03T20:26:40.464 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.159+0000 7f67427fc700 1 --2- 172.21.15.181:0/662069796 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f673803e590 0x7f6738040a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:40.464 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.159+0000 7f67427fc700 1 -- 172.21.15.181:0/662069796 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6748053d70 con 0x7f674c109a10 2024-05-03T20:26:40.464 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.159+0000 7f67508d6700 1 --2- 172.21.15.181:0/662069796 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f673803e590 0x7f6738040a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:40.464 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.159+0000 7f67508d6700 1 --2- 172.21.15.181:0/662069796 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f673803e590 0x7f6738040a50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f673c006fd0 tx=0x7f673c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:40.465 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.164+0000 7f67427fc700 1 -- 172.21.15.181:0/662069796 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6748058070 con 0x7f674c109a10 2024-05-03T20:26:40.465 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.371+0000 7f675333b700 1 -- 172.21.15.181:0/662069796 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f674c108180 con 0x7f674c109a10 2024-05-03T20:26:40.465 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.371+0000 7f67427fc700 1 -- 172.21.15.181:0/662069796 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f674801f030 con 0x7f674c109a10 2024-05-03T20:26:40.465 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.374+0000 7f675333b700 1 -- 172.21.15.181:0/662069796 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f673803e590 msgr2=0x7f6738040a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:40.465 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.374+0000 7f675333b700 1 --2- 172.21.15.181:0/662069796 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f673803e590 0x7f6738040a50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f673c006fd0 tx=0x7f673c006e40 comp rx=0 tx=0).stop 2024-05-03T20:26:40.465 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.374+0000 7f675333b700 1 -- 172.21.15.181:0/662069796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f674c109a10 msgr2=0x7f674c109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:40.465 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.374+0000 7f675333b700 1 --2- 172.21.15.181:0/662069796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f674c109a10 0x7f674c109290 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f67480116e0 tx=0x7f6748011aa0 comp rx=0 tx=0).stop 2024-05-03T20:26:40.465 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.374+0000 7f675333b700 1 -- 172.21.15.181:0/662069796 shutdown_connections 2024-05-03T20:26:40.465 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.374+0000 7f675333b700 1 --2- 172.21.15.181:0/662069796 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f673803e590 0x7f6738040a50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:40.465 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.374+0000 7f675333b700 1 --2- 172.21.15.181:0/662069796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f674c109a10 0x7f674c109290 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:40.465 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.374+0000 7f675333b700 1 -- 172.21.15.181:0/662069796 >> 172.21.15.181:0/662069796 conn(0x7f674c103460 msgr2=0x7f674c103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:40.465 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.374+0000 7f675333b700 1 -- 172.21.15.181:0/662069796 shutdown_connections 2024-05-03T20:26:40.466 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:39.374+0000 7f675333b700 1 -- 172.21.15.181:0/662069796 wait complete. 2024-05-03T20:26:40.469 INFO:teuthology.orchestra.run.smithi181.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-03T20:26:40.469 INFO:teuthology.orchestra.run.smithi181.stdout:systemctl: stdout disabled 2024-05-03T20:26:40.476 INFO:teuthology.orchestra.run.smithi181.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-03T20:26:40.476 INFO:teuthology.orchestra.run.smithi181.stdout:systemctl: stdout inactive 2024-05-03T20:26:40.476 INFO:teuthology.orchestra.run.smithi181.stdout:firewalld.service is not enabled 2024-05-03T20:26:40.477 INFO:teuthology.orchestra.run.smithi181.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-03T20:26:40.478 INFO:teuthology.orchestra.run.smithi181.stdout:Ceph Dashboard is now available at: 2024-05-03T20:26:40.478 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:26:40.478 INFO:teuthology.orchestra.run.smithi181.stdout: URL: https://smithi181:8443/ 2024-05-03T20:26:40.478 INFO:teuthology.orchestra.run.smithi181.stdout: User: admin 2024-05-03T20:26:40.478 INFO:teuthology.orchestra.run.smithi181.stdout: Password: t3x10l9fjw 2024-05-03T20:26:40.478 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:26:40.478 INFO:teuthology.orchestra.run.smithi181.stdout:Saving cluster configuration to /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/config directory 2024-05-03T20:26:40.479 INFO:teuthology.orchestra.run.smithi181.stdout:Enabling autotune for osd_memory_target 2024-05-03T20:26:41.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:41 smithi181 ceph-mon[29801]: mgrmap e13: x(active, since 6s) 2024-05-03T20:26:41.725 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.911+0000 7fbfe9ef4700 1 Processor -- start 2024-05-03T20:26:41.725 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.911+0000 7fbfe9ef4700 1 -- start start 2024-05-03T20:26:41.725 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.911+0000 7fbfe9ef4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe4108f00 0x7fbfe4109340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:41.725 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.911+0000 7fbfe9ef4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe4109880 con 0x7fbfe4108f00 2024-05-03T20:26:41.725 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.911+0000 7fbfe37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe4108f00 0x7fbfe4109340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:41.725 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.911+0000 7fbfe37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe4108f00 0x7fbfe4109340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60246/0 (socket says 172.21.15.181:60246) 2024-05-03T20:26:41.725 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.911+0000 7fbfe37fe700 1 -- 172.21.15.181:0/721027098 learned_addr learned my addr 172.21.15.181:0/721027098 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:41.725 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.911+0000 7fbfe37fe700 1 -- 172.21.15.181:0/721027098 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe410a0e0 con 0x7fbfe4108f00 2024-05-03T20:26:41.726 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.912+0000 7fbfe37fe700 1 --2- 172.21.15.181:0/721027098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe4108f00 0x7fbfe4109340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbfd40098a0 tx=0x7fbfd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e41f43d9f0dcb9e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:41.726 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.912+0000 7fbfe27fc700 1 -- 172.21.15.181:0/721027098 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfd4004160 con 0x7fbfe4108f00 2024-05-03T20:26:41.726 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.912+0000 7fbfe27fc700 1 -- 172.21.15.181:0/721027098 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfd4004300 con 0x7fbfe4108f00 2024-05-03T20:26:41.726 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.912+0000 7fbfe27fc700 1 -- 172.21.15.181:0/721027098 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfd4011a20 con 0x7fbfe4108f00 2024-05-03T20:26:41.726 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.912+0000 7fbfe9ef4700 1 -- 172.21.15.181:0/721027098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe4108f00 msgr2=0x7fbfe4109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:41.726 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.912+0000 7fbfe9ef4700 1 --2- 172.21.15.181:0/721027098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe4108f00 0x7fbfe4109340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbfd40098a0 tx=0x7fbfd4009bb0 comp rx=0 tx=0).stop 2024-05-03T20:26:41.726 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.912+0000 7fbfe9ef4700 1 -- 172.21.15.181:0/721027098 shutdown_connections 2024-05-03T20:26:41.726 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.912+0000 7fbfe9ef4700 1 --2- 172.21.15.181:0/721027098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe4108f00 0x7fbfe4109340 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:41.726 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.912+0000 7fbfe9ef4700 1 -- 172.21.15.181:0/721027098 >> 172.21.15.181:0/721027098 conn(0x7fbfe4104510 msgr2=0x7fbfe4106930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:41.726 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.912+0000 7fbfe9ef4700 1 -- 172.21.15.181:0/721027098 shutdown_connections 2024-05-03T20:26:41.726 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.912+0000 7fbfe9ef4700 1 -- 172.21.15.181:0/721027098 wait complete. 2024-05-03T20:26:41.726 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.913+0000 7fbfe9ef4700 1 Processor -- start 2024-05-03T20:26:41.726 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.913+0000 7fbfe9ef4700 1 -- start start 2024-05-03T20:26:41.726 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.913+0000 7fbfe9ef4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe4108f00 0x7fbfe407eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:41.727 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.913+0000 7fbfe9ef4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe410d3a0 con 0x7fbfe4108f00 2024-05-03T20:26:41.727 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.914+0000 7fbfe37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe4108f00 0x7fbfe407eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:41.727 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.914+0000 7fbfe37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe4108f00 0x7fbfe407eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60254/0 (socket says 172.21.15.181:60254) 2024-05-03T20:26:41.727 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.914+0000 7fbfe37fe700 1 -- 172.21.15.181:0/824593473 learned_addr learned my addr 172.21.15.181:0/824593473 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:41.727 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.914+0000 7fbfe37fe700 1 -- 172.21.15.181:0/824593473 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe407f090 con 0x7fbfe4108f00 2024-05-03T20:26:41.727 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.914+0000 7fbfe37fe700 1 --2- 172.21.15.181:0/824593473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe4108f00 0x7fbfe407eb50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbfd40116e0 tx=0x7fbfd4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:41.727 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.914+0000 7fbfe0ff9700 1 -- 172.21.15.181:0/824593473 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfd40039e0 con 0x7fbfe4108f00 2024-05-03T20:26:41.727 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.914+0000 7fbfe9ef4700 1 -- 172.21.15.181:0/824593473 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfe407d280 con 0x7fbfe4108f00 2024-05-03T20:26:41.727 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.914+0000 7fbfe0ff9700 1 -- 172.21.15.181:0/824593473 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfd401b8f0 con 0x7fbfe4108f00 2024-05-03T20:26:41.727 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.914+0000 7fbfe0ff9700 1 -- 172.21.15.181:0/824593473 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfd4023760 con 0x7fbfe4108f00 2024-05-03T20:26:41.727 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.914+0000 7fbfe9ef4700 1 -- 172.21.15.181:0/824593473 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfe407d7c0 con 0x7fbfe4108f00 2024-05-03T20:26:41.727 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.915+0000 7fbfe0ff9700 1 -- 172.21.15.181:0/824593473 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fbfd4023a00 con 0x7fbfe4108f00 2024-05-03T20:26:41.727 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.915+0000 7fbfe9ef4700 1 -- 172.21.15.181:0/824593473 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfe4064b00 con 0x7fbfe4108f00 2024-05-03T20:26:41.727 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.915+0000 7fbfe0ff9700 1 --2- 172.21.15.181:0/824593473 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fbfcc03e630 0x7fbfcc040af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:41.727 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.915+0000 7fbfe0ff9700 1 -- 172.21.15.181:0/824593473 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbfd4053ff0 con 0x7fbfe4108f00 2024-05-03T20:26:41.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.915+0000 7fbfe2ffd700 1 --2- 172.21.15.181:0/824593473 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fbfcc03e630 0x7fbfcc040af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:41.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.916+0000 7fbfe2ffd700 1 --2- 172.21.15.181:0/824593473 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fbfcc03e630 0x7fbfcc040af0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8006fd0 tx=0x7fbfd8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:41.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:40.920+0000 7fbfe0ff9700 1 -- 172.21.15.181:0/824593473 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fbfd401f030 con 0x7fbfe4108f00 2024-05-03T20:26:41.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:41.124+0000 7fbfe9ef4700 1 -- 172.21.15.181:0/824593473 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fbfe407da40 con 0x7fbfe4108f00 2024-05-03T20:26:41.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:41.124+0000 7fbfe0ff9700 1 -- 172.21.15.181:0/824593473 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fbfd4023df0 con 0x7fbfe4108f00 2024-05-03T20:26:41.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:41.126+0000 7fbfe9ef4700 1 -- 172.21.15.181:0/824593473 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fbfcc03e630 msgr2=0x7fbfcc040af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:41.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:41.126+0000 7fbfe9ef4700 1 --2- 172.21.15.181:0/824593473 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fbfcc03e630 0x7fbfcc040af0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8006fd0 tx=0x7fbfd8006e40 comp rx=0 tx=0).stop 2024-05-03T20:26:41.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:41.126+0000 7fbfe9ef4700 1 -- 172.21.15.181:0/824593473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe4108f00 msgr2=0x7fbfe407eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:41.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:41.126+0000 7fbfe9ef4700 1 --2- 172.21.15.181:0/824593473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe4108f00 0x7fbfe407eb50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbfd40116e0 tx=0x7fbfd4011aa0 comp rx=0 tx=0).stop 2024-05-03T20:26:41.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:41.127+0000 7fbfe9ef4700 1 -- 172.21.15.181:0/824593473 shutdown_connections 2024-05-03T20:26:41.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:41.127+0000 7fbfe9ef4700 1 --2- 172.21.15.181:0/824593473 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fbfcc03e630 0x7fbfcc040af0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:41.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:41.127+0000 7fbfe9ef4700 1 --2- 172.21.15.181:0/824593473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe4108f00 0x7fbfe407eb50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:41.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:41.127+0000 7fbfe9ef4700 1 -- 172.21.15.181:0/824593473 >> 172.21.15.181:0/824593473 conn(0x7fbfe4104510 msgr2=0x7fbfe4104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:41.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:41.127+0000 7fbfe9ef4700 1 -- 172.21.15.181:0/824593473 shutdown_connections 2024-05-03T20:26:41.728 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:41.127+0000 7fbfe9ef4700 1 -- 172.21.15.181:0/824593473 wait complete. 2024-05-03T20:26:43.099 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.144+0000 7feb370de700 1 Processor -- start 2024-05-03T20:26:43.099 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.144+0000 7feb370de700 1 -- start start 2024-05-03T20:26:43.099 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.144+0000 7feb370de700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb30109a80 0x7feb3010be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:43.100 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.144+0000 7feb370de700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feb3007ce30 con 0x7feb30109a80 2024-05-03T20:26:43.100 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.144+0000 7feb34e7a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb30109a80 0x7feb3010be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:43.100 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.144+0000 7feb34e7a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb30109a80 0x7feb3010be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60258/0 (socket says 172.21.15.181:60258) 2024-05-03T20:26:43.100 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.144+0000 7feb34e7a700 1 -- 172.21.15.181:0/1320887329 learned_addr learned my addr 172.21.15.181:0/1320887329 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:43.100 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.144+0000 7feb34e7a700 1 -- 172.21.15.181:0/1320887329 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb3010c3b0 con 0x7feb30109a80 2024-05-03T20:26:43.100 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.144+0000 7feb34e7a700 1 --2- 172.21.15.181:0/1320887329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb30109a80 0x7feb3010be70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feb240098a0 tx=0x7feb24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935b7762668a4cab server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:43.100 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.145+0000 7feb2f7fe700 1 -- 172.21.15.181:0/1320887329 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb24004160 con 0x7feb30109a80 2024-05-03T20:26:43.100 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.145+0000 7feb2f7fe700 1 -- 172.21.15.181:0/1320887329 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb24004300 con 0x7feb30109a80 2024-05-03T20:26:43.100 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.145+0000 7feb2f7fe700 1 -- 172.21.15.181:0/1320887329 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb24011a20 con 0x7feb30109a80 2024-05-03T20:26:43.100 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.145+0000 7feb370de700 1 -- 172.21.15.181:0/1320887329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb30109a80 msgr2=0x7feb3010be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:43.100 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.145+0000 7feb370de700 1 --2- 172.21.15.181:0/1320887329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb30109a80 0x7feb3010be70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feb240098a0 tx=0x7feb24009bb0 comp rx=0 tx=0).stop 2024-05-03T20:26:43.100 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.145+0000 7feb370de700 1 -- 172.21.15.181:0/1320887329 shutdown_connections 2024-05-03T20:26:43.100 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.145+0000 7feb370de700 1 --2- 172.21.15.181:0/1320887329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb30109a80 0x7feb3010be70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:43.100 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.145+0000 7feb370de700 1 -- 172.21.15.181:0/1320887329 >> 172.21.15.181:0/1320887329 conn(0x7feb30103490 msgr2=0x7feb301058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:43.100 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.145+0000 7feb370de700 1 -- 172.21.15.181:0/1320887329 shutdown_connections 2024-05-03T20:26:43.100 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.145+0000 7feb370de700 1 -- 172.21.15.181:0/1320887329 wait complete. 2024-05-03T20:26:43.101 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.146+0000 7feb370de700 1 Processor -- start 2024-05-03T20:26:43.101 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.146+0000 7feb370de700 1 -- start start 2024-05-03T20:26:43.101 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.146+0000 7feb370de700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb30109a80 0x7feb30109300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:43.101 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.146+0000 7feb370de700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feb3010d360 con 0x7feb30109a80 2024-05-03T20:26:43.101 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.146+0000 7feb34e7a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb30109a80 0x7feb30109300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:43.101 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.146+0000 7feb34e7a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb30109a80 0x7feb30109300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60272/0 (socket says 172.21.15.181:60272) 2024-05-03T20:26:43.101 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.146+0000 7feb34e7a700 1 -- 172.21.15.181:0/2173111626 learned_addr learned my addr 172.21.15.181:0/2173111626 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:43.101 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.146+0000 7feb34e7a700 1 -- 172.21.15.181:0/2173111626 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb30109840 con 0x7feb30109a80 2024-05-03T20:26:43.101 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.147+0000 7feb34e7a700 1 --2- 172.21.15.181:0/2173111626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb30109a80 0x7feb30109300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feb24004660 tx=0x7feb24011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:43.101 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.147+0000 7feb2dffb700 1 -- 172.21.15.181:0/2173111626 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb24003aa0 con 0x7feb30109a80 2024-05-03T20:26:43.101 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.147+0000 7feb2dffb700 1 -- 172.21.15.181:0/2173111626 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb2401b8f0 con 0x7feb30109a80 2024-05-03T20:26:43.101 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.147+0000 7feb2dffb700 1 -- 172.21.15.181:0/2173111626 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb24023900 con 0x7feb30109a80 2024-05-03T20:26:43.101 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.147+0000 7feb370de700 1 -- 172.21.15.181:0/2173111626 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb30107a30 con 0x7feb30109a80 2024-05-03T20:26:43.101 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.147+0000 7feb370de700 1 -- 172.21.15.181:0/2173111626 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb30107f70 con 0x7feb30109a80 2024-05-03T20:26:43.102 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.148+0000 7feb2dffb700 1 -- 172.21.15.181:0/2173111626 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7feb2401baa0 con 0x7feb30109a80 2024-05-03T20:26:43.102 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.148+0000 7feb370de700 1 -- 172.21.15.181:0/2173111626 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb30064b00 con 0x7feb30109a80 2024-05-03T20:26:43.102 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.148+0000 7feb2dffb700 1 --2- 172.21.15.181:0/2173111626 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7feb2003e680 0x7feb20040b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:43.102 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.148+0000 7feb2dffb700 1 -- 172.21.15.181:0/2173111626 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7feb24053f50 con 0x7feb30109a80 2024-05-03T20:26:43.102 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.148+0000 7feb2ffff700 1 --2- 172.21.15.181:0/2173111626 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7feb2003e680 0x7feb20040b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:43.102 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.149+0000 7feb2ffff700 1 --2- 172.21.15.181:0/2173111626 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7feb2003e680 0x7feb20040b40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7feb1c006fd0 tx=0x7feb1c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:43.102 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.153+0000 7feb2dffb700 1 -- 172.21.15.181:0/2173111626 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7feb2401f030 con 0x7feb30109a80 2024-05-03T20:26:43.102 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.475+0000 7feb370de700 1 -- 172.21.15.181:0/2173111626 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7feb301081f0 con 0x7feb30109a80 2024-05-03T20:26:43.102 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.478+0000 7feb2dffb700 1 -- 172.21.15.181:0/2173111626 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7feb24018350 con 0x7feb30109a80 2024-05-03T20:26:43.102 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.481+0000 7feb370de700 1 -- 172.21.15.181:0/2173111626 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7feb2003e680 msgr2=0x7feb20040b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:43.102 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.481+0000 7feb370de700 1 --2- 172.21.15.181:0/2173111626 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7feb2003e680 0x7feb20040b40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7feb1c006fd0 tx=0x7feb1c006e40 comp rx=0 tx=0).stop 2024-05-03T20:26:43.102 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.481+0000 7feb370de700 1 -- 172.21.15.181:0/2173111626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb30109a80 msgr2=0x7feb30109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:43.102 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.481+0000 7feb370de700 1 --2- 172.21.15.181:0/2173111626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb30109a80 0x7feb30109300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feb24004660 tx=0x7feb24011aa0 comp rx=0 tx=0).stop 2024-05-03T20:26:43.102 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.481+0000 7feb370de700 1 -- 172.21.15.181:0/2173111626 shutdown_connections 2024-05-03T20:26:43.103 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.481+0000 7feb370de700 1 --2- 172.21.15.181:0/2173111626 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7feb2003e680 0x7feb20040b40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:43.103 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.481+0000 7feb370de700 1 --2- 172.21.15.181:0/2173111626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb30109a80 0x7feb30109300 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:43.103 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.481+0000 7feb370de700 1 -- 172.21.15.181:0/2173111626 >> 172.21.15.181:0/2173111626 conn(0x7feb30103490 msgr2=0x7feb30103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:43.103 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.481+0000 7feb370de700 1 -- 172.21.15.181:0/2173111626 shutdown_connections 2024-05-03T20:26:43.103 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr 2024-05-03T20:26:42.481+0000 7feb370de700 1 -- 172.21.15.181:0/2173111626 wait complete. 2024-05-03T20:26:43.103 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-03T20:26:43.103 INFO:teuthology.orchestra.run.smithi181.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-03T20:26:43.103 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:26:43.103 INFO:teuthology.orchestra.run.smithi181.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-03T20:26:43.103 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:26:43.103 INFO:teuthology.orchestra.run.smithi181.stdout:Or, if you are only running a single cluster on this host: 2024-05-03T20:26:43.103 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:26:43.103 INFO:teuthology.orchestra.run.smithi181.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-03T20:26:43.103 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:26:43.104 INFO:teuthology.orchestra.run.smithi181.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-03T20:26:43.104 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:26:43.104 INFO:teuthology.orchestra.run.smithi181.stdout: ceph telemetry on 2024-05-03T20:26:43.104 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:26:43.104 INFO:teuthology.orchestra.run.smithi181.stdout:For more information see: 2024-05-03T20:26:43.104 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:26:43.104 INFO:teuthology.orchestra.run.smithi181.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-03T20:26:43.104 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:26:43.104 INFO:teuthology.orchestra.run.smithi181.stdout:Bootstrap complete. 2024-05-03T20:26:43.134 INFO:tasks.cephadm:Fetching config... 2024-05-03T20:26:43.135 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-03T20:26:43.135 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-03T20:26:43.150 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-03T20:26:43.150 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-03T20:26:43.150 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-03T20:26:43.205 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-03T20:26:43.205 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-03T20:26:43.205 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-03T20:26:43.269 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-03T20:26:43.269 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-03T20:26:43.269 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-03T20:26:43.325 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-03T20:26:43.325 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjGsDBfffgbt1QsmUENw/TRXe6U88W3TMV1ywiD495F1U/QGhgm/+nrec+Dj+gLiZ9OafbMUNLmpt+8uQ13L8Nyx6gK+msTLGd0Mhh7AhJBy3qUDaGkU18uCnsP1w7eJUsU6TBoD34n7TYGClM/oM+0vpxhPC3iRo+rZbkaC364JiNQWumxkRCp84GhOl2nTeFp8fq5V4pst5xkrTfshcjes0mkrnnDFD+Y91Giu1KFnWfQvT2olvg5ryddALxw3sVpi+7wjNQzBVHQHn2BbJo52PVMtCGI3qw8Z9WSwniQ86C4YNTqzU3cIwY6iZIPSMYaUvG8bNyNJzZUkdBv3MjbFJMfIOqQY1n64uzPR5xhKL5K4M8sbh0kzfU4gwSRHJ6mHIq7HzahvhiA7tFTvdKcMxP6BQsVQIdudN7RHKFAmrLcsf1y/+mefuuLhFqHfRB333X6kG+i8dSR6s9PkkjSsivNQDTNzBEjgyIC6snvxpSR1qYfSvEK+7DFrXIt30= ceph-1cd058ee-098b-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-03T20:26:43.412 INFO:teuthology.orchestra.run.smithi181.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjGsDBfffgbt1QsmUENw/TRXe6U88W3TMV1ywiD495F1U/QGhgm/+nrec+Dj+gLiZ9OafbMUNLmpt+8uQ13L8Nyx6gK+msTLGd0Mhh7AhJBy3qUDaGkU18uCnsP1w7eJUsU6TBoD34n7TYGClM/oM+0vpxhPC3iRo+rZbkaC364JiNQWumxkRCp84GhOl2nTeFp8fq5V4pst5xkrTfshcjes0mkrnnDFD+Y91Giu1KFnWfQvT2olvg5ryddALxw3sVpi+7wjNQzBVHQHn2BbJo52PVMtCGI3qw8Z9WSwniQ86C4YNTqzU3cIwY6iZIPSMYaUvG8bNyNJzZUkdBv3MjbFJMfIOqQY1n64uzPR5xhKL5K4M8sbh0kzfU4gwSRHJ6mHIq7HzahvhiA7tFTvdKcMxP6BQsVQIdudN7RHKFAmrLcsf1y/+mefuuLhFqHfRB333X6kG+i8dSR6s9PkkjSsivNQDTNzBEjgyIC6snvxpSR1qYfSvEK+7DFrXIt30= ceph-1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:26:43.422 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjGsDBfffgbt1QsmUENw/TRXe6U88W3TMV1ywiD495F1U/QGhgm/+nrec+Dj+gLiZ9OafbMUNLmpt+8uQ13L8Nyx6gK+msTLGd0Mhh7AhJBy3qUDaGkU18uCnsP1w7eJUsU6TBoD34n7TYGClM/oM+0vpxhPC3iRo+rZbkaC364JiNQWumxkRCp84GhOl2nTeFp8fq5V4pst5xkrTfshcjes0mkrnnDFD+Y91Giu1KFnWfQvT2olvg5ryddALxw3sVpi+7wjNQzBVHQHn2BbJo52PVMtCGI3qw8Z9WSwniQ86C4YNTqzU3cIwY6iZIPSMYaUvG8bNyNJzZUkdBv3MjbFJMfIOqQY1n64uzPR5xhKL5K4M8sbh0kzfU4gwSRHJ6mHIq7HzahvhiA7tFTvdKcMxP6BQsVQIdudN7RHKFAmrLcsf1y/+mefuuLhFqHfRB333X6kG+i8dSR6s9PkkjSsivNQDTNzBEjgyIC6snvxpSR1qYfSvEK+7DFrXIt30= ceph-1cd058ee-098b-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-03T20:26:43.462 INFO:teuthology.orchestra.run.smithi196.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjGsDBfffgbt1QsmUENw/TRXe6U88W3TMV1ywiD495F1U/QGhgm/+nrec+Dj+gLiZ9OafbMUNLmpt+8uQ13L8Nyx6gK+msTLGd0Mhh7AhJBy3qUDaGkU18uCnsP1w7eJUsU6TBoD34n7TYGClM/oM+0vpxhPC3iRo+rZbkaC364JiNQWumxkRCp84GhOl2nTeFp8fq5V4pst5xkrTfshcjes0mkrnnDFD+Y91Giu1KFnWfQvT2olvg5ryddALxw3sVpi+7wjNQzBVHQHn2BbJo52PVMtCGI3qw8Z9WSwniQ86C4YNTqzU3cIwY6iZIPSMYaUvG8bNyNJzZUkdBv3MjbFJMfIOqQY1n64uzPR5xhKL5K4M8sbh0kzfU4gwSRHJ6mHIq7HzahvhiA7tFTvdKcMxP6BQsVQIdudN7RHKFAmrLcsf1y/+mefuuLhFqHfRB333X6kG+i8dSR6s9PkkjSsivNQDTNzBEjgyIC6snvxpSR1qYfSvEK+7DFrXIt30= ceph-1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:26:43.473 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjGsDBfffgbt1QsmUENw/TRXe6U88W3TMV1ywiD495F1U/QGhgm/+nrec+Dj+gLiZ9OafbMUNLmpt+8uQ13L8Nyx6gK+msTLGd0Mhh7AhJBy3qUDaGkU18uCnsP1w7eJUsU6TBoD34n7TYGClM/oM+0vpxhPC3iRo+rZbkaC364JiNQWumxkRCp84GhOl2nTeFp8fq5V4pst5xkrTfshcjes0mkrnnDFD+Y91Giu1KFnWfQvT2olvg5ryddALxw3sVpi+7wjNQzBVHQHn2BbJo52PVMtCGI3qw8Z9WSwniQ86C4YNTqzU3cIwY6iZIPSMYaUvG8bNyNJzZUkdBv3MjbFJMfIOqQY1n64uzPR5xhKL5K4M8sbh0kzfU4gwSRHJ6mHIq7HzahvhiA7tFTvdKcMxP6BQsVQIdudN7RHKFAmrLcsf1y/+mefuuLhFqHfRB333X6kG+i8dSR6s9PkkjSsivNQDTNzBEjgyIC6snvxpSR1qYfSvEK+7DFrXIt30= ceph-1cd058ee-098b-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-03T20:26:43.513 INFO:teuthology.orchestra.run.smithi204.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjGsDBfffgbt1QsmUENw/TRXe6U88W3TMV1ywiD495F1U/QGhgm/+nrec+Dj+gLiZ9OafbMUNLmpt+8uQ13L8Nyx6gK+msTLGd0Mhh7AhJBy3qUDaGkU18uCnsP1w7eJUsU6TBoD34n7TYGClM/oM+0vpxhPC3iRo+rZbkaC364JiNQWumxkRCp84GhOl2nTeFp8fq5V4pst5xkrTfshcjes0mkrnnDFD+Y91Giu1KFnWfQvT2olvg5ryddALxw3sVpi+7wjNQzBVHQHn2BbJo52PVMtCGI3qw8Z9WSwniQ86C4YNTqzU3cIwY6iZIPSMYaUvG8bNyNJzZUkdBv3MjbFJMfIOqQY1n64uzPR5xhKL5K4M8sbh0kzfU4gwSRHJ6mHIq7HzahvhiA7tFTvdKcMxP6BQsVQIdudN7RHKFAmrLcsf1y/+mefuuLhFqHfRB333X6kG+i8dSR6s9PkkjSsivNQDTNzBEjgyIC6snvxpSR1qYfSvEK+7DFrXIt30= ceph-1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:26:43.524 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-03T20:26:43.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:43 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/2173111626' entity='client.admin' 2024-05-03T20:26:43.767 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:26:46.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.154+0000 7f5f33310700 1 -- 172.21.15.181:0/231330852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f2c101e00 msgr2=0x7f5f2c102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:46.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.154+0000 7f5f33310700 1 --2- 172.21.15.181:0/231330852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f2c101e00 0x7f5f2c102220 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009910 tx=0x7f5f1c009c20 comp rx=0 tx=0).stop 2024-05-03T20:26:46.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.154+0000 7f5f33310700 1 -- 172.21.15.181:0/231330852 shutdown_connections 2024-05-03T20:26:46.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.154+0000 7f5f33310700 1 --2- 172.21.15.181:0/231330852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f2c101e00 0x7f5f2c102220 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:46.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.154+0000 7f5f33310700 1 -- 172.21.15.181:0/231330852 >> 172.21.15.181:0/231330852 conn(0x7f5f2c0fd3a0 msgr2=0x7f5f2c0ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:46.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.154+0000 7f5f33310700 1 -- 172.21.15.181:0/231330852 shutdown_connections 2024-05-03T20:26:46.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.154+0000 7f5f33310700 1 -- 172.21.15.181:0/231330852 wait complete. 2024-05-03T20:26:46.157 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.155+0000 7f5f33310700 1 Processor -- start 2024-05-03T20:26:46.157 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.155+0000 7f5f33310700 1 -- start start 2024-05-03T20:26:46.157 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.155+0000 7f5f33310700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f2c101e00 0x7f5f2c197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:46.157 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.155+0000 7f5f33310700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c107cc0 con 0x7f5f2c101e00 2024-05-03T20:26:46.157 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.155+0000 7f5f310ac700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f2c101e00 0x7f5f2c197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:46.157 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.155+0000 7f5f310ac700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f2c101e00 0x7f5f2c197490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51900/0 (socket says 172.21.15.181:51900) 2024-05-03T20:26:46.157 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.155+0000 7f5f310ac700 1 -- 172.21.15.181:0/437026299 learned_addr learned my addr 172.21.15.181:0/437026299 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:46.157 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.156+0000 7f5f310ac700 1 -- 172.21.15.181:0/437026299 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2c1979d0 con 0x7f5f2c101e00 2024-05-03T20:26:46.157 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.156+0000 7f5f310ac700 1 --2- 172.21.15.181:0/437026299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f2c101e00 0x7f5f2c197490 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c005b40 tx=0x7f5f1c004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:46.157 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.156+0000 7f5f227fc700 1 -- 172.21.15.181:0/437026299 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f1c027070 con 0x7f5f2c101e00 2024-05-03T20:26:46.157 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.156+0000 7f5f33310700 1 -- 172.21.15.181:0/437026299 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f2c197c60 con 0x7f5f2c101e00 2024-05-03T20:26:46.157 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.156+0000 7f5f227fc700 1 -- 172.21.15.181:0/437026299 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f1c00be10 con 0x7f5f2c101e00 2024-05-03T20:26:46.157 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.156+0000 7f5f227fc700 1 -- 172.21.15.181:0/437026299 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f1c022de0 con 0x7f5f2c101e00 2024-05-03T20:26:46.157 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.156+0000 7f5f33310700 1 -- 172.21.15.181:0/437026299 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f2c19a950 con 0x7f5f2c101e00 2024-05-03T20:26:46.158 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.157+0000 7f5f227fc700 1 -- 172.21.15.181:0/437026299 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f5f1c02a070 con 0x7f5f2c101e00 2024-05-03T20:26:46.158 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.157+0000 7f5f33310700 1 -- 172.21.15.181:0/437026299 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f2c108440 con 0x7f5f2c101e00 2024-05-03T20:26:46.158 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.157+0000 7f5f227fc700 1 --2- 172.21.15.181:0/437026299 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5f1803e510 0x7f5f180409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:46.159 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.157+0000 7f5f227fc700 1 -- 172.21.15.181:0/437026299 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5f1c01e030 con 0x7f5f2c101e00 2024-05-03T20:26:46.159 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.157+0000 7f5f308ab700 1 --2- 172.21.15.181:0/437026299 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5f1803e510 0x7f5f180409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:46.159 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.158+0000 7f5f308ab700 1 --2- 172.21.15.181:0/437026299 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5f1803e510 0x7f5f180409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5f28006fd0 tx=0x7f5f28006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:46.162 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.162+0000 7f5f227fc700 1 -- 172.21.15.181:0/437026299 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5f1c013070 con 0x7f5f2c101e00 2024-05-03T20:26:46.370 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.370+0000 7f5f33310700 1 -- 172.21.15.181:0/437026299 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f5f2c02e620 con 0x7f5f2c101e00 2024-05-03T20:26:46.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.372+0000 7f5f227fc700 1 -- 172.21.15.181:0/437026299 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f5f1c031020 con 0x7f5f2c101e00 2024-05-03T20:26:46.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.375+0000 7f5f33310700 1 -- 172.21.15.181:0/437026299 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5f1803e510 msgr2=0x7f5f180409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:46.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.375+0000 7f5f33310700 1 --2- 172.21.15.181:0/437026299 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5f1803e510 0x7f5f180409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5f28006fd0 tx=0x7f5f28006e40 comp rx=0 tx=0).stop 2024-05-03T20:26:46.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.375+0000 7f5f33310700 1 -- 172.21.15.181:0/437026299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f2c101e00 msgr2=0x7f5f2c197490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:46.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.375+0000 7f5f33310700 1 --2- 172.21.15.181:0/437026299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f2c101e00 0x7f5f2c197490 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c005b40 tx=0x7f5f1c004f80 comp rx=0 tx=0).stop 2024-05-03T20:26:46.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.375+0000 7f5f33310700 1 -- 172.21.15.181:0/437026299 shutdown_connections 2024-05-03T20:26:46.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.375+0000 7f5f33310700 1 --2- 172.21.15.181:0/437026299 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5f1803e510 0x7f5f180409d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:46.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.375+0000 7f5f33310700 1 --2- 172.21.15.181:0/437026299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f2c101e00 0x7f5f2c197490 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:46.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.375+0000 7f5f33310700 1 -- 172.21.15.181:0/437026299 >> 172.21.15.181:0/437026299 conn(0x7f5f2c0fd3a0 msgr2=0x7f5f2c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:46.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.375+0000 7f5f33310700 1 -- 172.21.15.181:0/437026299 shutdown_connections 2024-05-03T20:26:46.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:46.375+0000 7f5f33310700 1 -- 172.21.15.181:0/437026299 wait complete. 2024-05-03T20:26:47.042 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-03T20:26:47.042 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-03T20:26:47.610 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:26:47.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:47 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/437026299' entity='client.admin' 2024-05-03T20:26:48.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:26:48.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:26:48.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi181", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:26:48.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:26:48.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:26:48.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:26:49.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.091+0000 7fad21341700 1 -- 172.21.15.181:0/945962442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad1c103a70 msgr2=0x7fad1c105e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:49.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.091+0000 7fad21341700 1 --2- 172.21.15.181:0/945962442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad1c103a70 0x7fad1c105e60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fad0c009910 tx=0x7fad0c009c20 comp rx=0 tx=0).stop 2024-05-03T20:26:49.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.091+0000 7fad21341700 1 -- 172.21.15.181:0/945962442 shutdown_connections 2024-05-03T20:26:49.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.091+0000 7fad21341700 1 --2- 172.21.15.181:0/945962442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad1c103a70 0x7fad1c105e60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:49.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.091+0000 7fad21341700 1 -- 172.21.15.181:0/945962442 >> 172.21.15.181:0/945962442 conn(0x7fad1c0fd400 msgr2=0x7fad1c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:49.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.091+0000 7fad21341700 1 -- 172.21.15.181:0/945962442 shutdown_connections 2024-05-03T20:26:49.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.091+0000 7fad21341700 1 -- 172.21.15.181:0/945962442 wait complete. 2024-05-03T20:26:49.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.091+0000 7fad21341700 1 Processor -- start 2024-05-03T20:26:49.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.091+0000 7fad21341700 1 -- start start 2024-05-03T20:26:49.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.092+0000 7fad21341700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad1c103a70 0x7fad1c198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:49.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.092+0000 7fad21341700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c108e80 con 0x7fad1c103a70 2024-05-03T20:26:49.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.092+0000 7fad1affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad1c103a70 0x7fad1c198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:49.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.092+0000 7fad1affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad1c103a70 0x7fad1c198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51924/0 (socket says 172.21.15.181:51924) 2024-05-03T20:26:49.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.092+0000 7fad1affd700 1 -- 172.21.15.181:0/415185743 learned_addr learned my addr 172.21.15.181:0/415185743 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:49.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.092+0000 7fad1affd700 1 -- 172.21.15.181:0/415185743 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1c198ab0 con 0x7fad1c103a70 2024-05-03T20:26:49.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.092+0000 7fad1affd700 1 --2- 172.21.15.181:0/415185743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad1c103a70 0x7fad1c198570 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fad0c005b40 tx=0x7fad0c004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:49.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.092+0000 7fad18ff9700 1 -- 172.21.15.181:0/415185743 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad0c027070 con 0x7fad1c103a70 2024-05-03T20:26:49.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.092+0000 7fad18ff9700 1 -- 172.21.15.181:0/415185743 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad0c00be10 con 0x7fad1c103a70 2024-05-03T20:26:49.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.092+0000 7fad21341700 1 -- 172.21.15.181:0/415185743 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad1c198d40 con 0x7fad1c103a70 2024-05-03T20:26:49.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.092+0000 7fad18ff9700 1 -- 172.21.15.181:0/415185743 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad0c022d80 con 0x7fad1c103a70 2024-05-03T20:26:49.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.092+0000 7fad21341700 1 -- 172.21.15.181:0/415185743 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad1c19ba30 con 0x7fad1c103a70 2024-05-03T20:26:49.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.093+0000 7fad18ff9700 1 -- 172.21.15.181:0/415185743 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fad0c02a070 con 0x7fad1c103a70 2024-05-03T20:26:49.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.093+0000 7fad21341700 1 -- 172.21.15.181:0/415185743 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad1c109600 con 0x7fad1c103a70 2024-05-03T20:26:49.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.093+0000 7fad18ff9700 1 --2- 172.21.15.181:0/415185743 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fad0803e570 0x7fad08040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:49.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.093+0000 7fad18ff9700 1 -- 172.21.15.181:0/415185743 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fad0c01e030 con 0x7fad1c103a70 2024-05-03T20:26:49.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.094+0000 7fad127fc700 1 --2- 172.21.15.181:0/415185743 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fad0803e570 0x7fad08040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:49.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.094+0000 7fad127fc700 1 --2- 172.21.15.181:0/415185743 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fad0803e570 0x7fad08040a30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fad04006fd0 tx=0x7fad04006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:49.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.098+0000 7fad18ff9700 1 -- 172.21.15.181:0/415185743 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fad0c013070 con 0x7fad1c103a70 2024-05-03T20:26:49.304 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.304+0000 7fad21341700 1 -- 172.21.15.181:0/415185743 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fad1c02e620 con 0x7fad0803e570 2024-05-03T20:26:49.307 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.307+0000 7fad18ff9700 1 -- 172.21.15.181:0/415185743 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fad1c02e620 con 0x7fad0803e570 2024-05-03T20:26:49.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.309+0000 7fad21341700 1 -- 172.21.15.181:0/415185743 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fad0803e570 msgr2=0x7fad08040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:49.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.310+0000 7fad21341700 1 --2- 172.21.15.181:0/415185743 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fad0803e570 0x7fad08040a30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fad04006fd0 tx=0x7fad04006e40 comp rx=0 tx=0).stop 2024-05-03T20:26:49.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.310+0000 7fad21341700 1 -- 172.21.15.181:0/415185743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad1c103a70 msgr2=0x7fad1c198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:49.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.310+0000 7fad21341700 1 --2- 172.21.15.181:0/415185743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad1c103a70 0x7fad1c198570 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fad0c005b40 tx=0x7fad0c004f80 comp rx=0 tx=0).stop 2024-05-03T20:26:49.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.310+0000 7fad21341700 1 -- 172.21.15.181:0/415185743 shutdown_connections 2024-05-03T20:26:49.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.310+0000 7fad21341700 1 --2- 172.21.15.181:0/415185743 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fad0803e570 0x7fad08040a30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:49.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.310+0000 7fad21341700 1 --2- 172.21.15.181:0/415185743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad1c103a70 0x7fad1c198570 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:49.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.310+0000 7fad21341700 1 -- 172.21.15.181:0/415185743 >> 172.21.15.181:0/415185743 conn(0x7fad1c0fd400 msgr2=0x7fad1c0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:49.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.310+0000 7fad21341700 1 -- 172.21.15.181:0/415185743 shutdown_connections 2024-05-03T20:26:49.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:49.310+0000 7fad21341700 1 -- 172.21.15.181:0/415185743 wait complete. 2024-05-03T20:26:50.285 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi196 2024-05-03T20:26:50.285 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-03T20:26:50.285 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.conf 2024-05-03T20:26:50.305 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-03T20:26:50.305 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-03T20:26:50.362 INFO:tasks.cephadm:Adding host smithi196 to orchestrator... 2024-05-03T20:26:50.362 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph orch host add smithi196 2024-05-03T20:26:50.615 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:26:50.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:50 smithi181 ceph-mon[29801]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:26:50.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:50 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:26:50.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:50 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:26:50.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:50 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:26:50.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:50 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:26:50.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:50 smithi181 ceph-mon[29801]: Updating smithi181:/etc/ceph/ceph.conf 2024-05-03T20:26:50.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:50 smithi181 ceph-mon[29801]: Updating smithi181:/var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T20:26:50.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:50 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:26:50.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:50 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:26:50.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:50 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:26:51.674 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:51 smithi181 ceph-mon[29801]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-05-03T20:26:51.674 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:51 smithi181 ceph-mon[29801]: Updating smithi181:/var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T20:26:52.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.185+0000 7fac6c956700 1 -- 172.21.15.181:0/153170550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac681030f0 msgr2=0x7fac681034d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:52.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.185+0000 7fac6c956700 1 --2- 172.21.15.181:0/153170550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac681030f0 0x7fac681034d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fac5c009910 tx=0x7fac5c009c20 comp rx=0 tx=0).stop 2024-05-03T20:26:52.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.185+0000 7fac6c956700 1 -- 172.21.15.181:0/153170550 shutdown_connections 2024-05-03T20:26:52.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.185+0000 7fac6c956700 1 --2- 172.21.15.181:0/153170550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac681030f0 0x7fac681034d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:52.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.185+0000 7fac6c956700 1 -- 172.21.15.181:0/153170550 >> 172.21.15.181:0/153170550 conn(0x7fac680fe9c0 msgr2=0x7fac68100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:52.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.185+0000 7fac6c956700 1 -- 172.21.15.181:0/153170550 shutdown_connections 2024-05-03T20:26:52.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.185+0000 7fac6c956700 1 -- 172.21.15.181:0/153170550 wait complete. 2024-05-03T20:26:52.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.186+0000 7fac6c956700 1 Processor -- start 2024-05-03T20:26:52.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.186+0000 7fac6c956700 1 -- start start 2024-05-03T20:26:52.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.186+0000 7fac6c956700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac681030f0 0x7fac680757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:52.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.186+0000 7fac6c956700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fac6810d370 con 0x7fac681030f0 2024-05-03T20:26:52.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.186+0000 7fac6659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac681030f0 0x7fac680757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:52.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.186+0000 7fac6659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac681030f0 0x7fac680757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51956/0 (socket says 172.21.15.181:51956) 2024-05-03T20:26:52.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.186+0000 7fac6659c700 1 -- 172.21.15.181:0/488525101 learned_addr learned my addr 172.21.15.181:0/488525101 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:52.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.187+0000 7fac6659c700 1 -- 172.21.15.181:0/488525101 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac68077690 con 0x7fac681030f0 2024-05-03T20:26:52.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.187+0000 7fac6659c700 1 --2- 172.21.15.181:0/488525101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac681030f0 0x7fac680757a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fac5c006010 tx=0x7fac5c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:52.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.187+0000 7fac577fe700 1 -- 172.21.15.181:0/488525101 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac5c026070 con 0x7fac681030f0 2024-05-03T20:26:52.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.187+0000 7fac577fe700 1 -- 172.21.15.181:0/488525101 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac5c00bc00 con 0x7fac681030f0 2024-05-03T20:26:52.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.187+0000 7fac577fe700 1 -- 172.21.15.181:0/488525101 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac5c003910 con 0x7fac681030f0 2024-05-03T20:26:52.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.187+0000 7fac6c956700 1 -- 172.21.15.181:0/488525101 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac68075d40 con 0x7fac681030f0 2024-05-03T20:26:52.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.187+0000 7fac6c956700 1 -- 172.21.15.181:0/488525101 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac68076200 con 0x7fac681030f0 2024-05-03T20:26:52.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.188+0000 7fac577fe700 1 -- 172.21.15.181:0/488525101 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fac5c029070 con 0x7fac681030f0 2024-05-03T20:26:52.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.188+0000 7fac6c956700 1 -- 172.21.15.181:0/488525101 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac681045d0 con 0x7fac681030f0 2024-05-03T20:26:52.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.188+0000 7fac577fe700 1 --2- 172.21.15.181:0/488525101 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fac5003e570 0x7fac50040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:52.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.188+0000 7fac577fe700 1 -- 172.21.15.181:0/488525101 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fac5c01e070 con 0x7fac681030f0 2024-05-03T20:26:52.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.188+0000 7fac65d9b700 1 --2- 172.21.15.181:0/488525101 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fac5003e570 0x7fac50040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:52.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.189+0000 7fac65d9b700 1 --2- 172.21.15.181:0/488525101 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fac5003e570 0x7fac50040a30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fac58006fd0 tx=0x7fac58006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:52.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.194+0000 7fac577fe700 1 -- 172.21.15.181:0/488525101 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fac5c013070 con 0x7fac681030f0 2024-05-03T20:26:52.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:52.399+0000 7fac6c956700 1 -- 172.21.15.181:0/488525101 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}) -- 0x7fac68076490 con 0x7fac5003e570 2024-05-03T20:26:53.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:52 smithi181 ceph-mon[29801]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:26:54.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:53 smithi181 ceph-mon[29801]: Deploying cephadm binary to smithi196 2024-05-03T20:26:55.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:54 smithi181 ceph-mon[29801]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:26:56.558 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:56.558+0000 7fac577fe700 1 -- 172.21.15.181:0/488525101 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fac68076490 con 0x7fac5003e570 2024-05-03T20:26:56.558 INFO:teuthology.orchestra.run.smithi181.stdout:Added host 'smithi196' with addr '172.21.15.196' 2024-05-03T20:26:56.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:56.560+0000 7fac6c956700 1 -- 172.21.15.181:0/488525101 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fac5003e570 msgr2=0x7fac50040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:56.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:56.560+0000 7fac6c956700 1 --2- 172.21.15.181:0/488525101 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fac5003e570 0x7fac50040a30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fac58006fd0 tx=0x7fac58006e40 comp rx=0 tx=0).stop 2024-05-03T20:26:56.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:56.561+0000 7fac6c956700 1 -- 172.21.15.181:0/488525101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac681030f0 msgr2=0x7fac680757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:56.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:56.561+0000 7fac6c956700 1 --2- 172.21.15.181:0/488525101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac681030f0 0x7fac680757a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fac5c006010 tx=0x7fac5c005780 comp rx=0 tx=0).stop 2024-05-03T20:26:56.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:56.561+0000 7fac6c956700 1 -- 172.21.15.181:0/488525101 shutdown_connections 2024-05-03T20:26:56.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:56.561+0000 7fac6c956700 1 --2- 172.21.15.181:0/488525101 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fac5003e570 0x7fac50040a30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:56.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:56.561+0000 7fac6c956700 1 --2- 172.21.15.181:0/488525101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac681030f0 0x7fac680757a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:56.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:56.561+0000 7fac6c956700 1 -- 172.21.15.181:0/488525101 >> 172.21.15.181:0/488525101 conn(0x7fac680fe9c0 msgr2=0x7fac68104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:56.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:56.561+0000 7fac6c956700 1 -- 172.21.15.181:0/488525101 shutdown_connections 2024-05-03T20:26:56.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:56.561+0000 7fac6c956700 1 -- 172.21.15.181:0/488525101 wait complete. 2024-05-03T20:26:57.255 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-03T20:26:57.492 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:26:57.816 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:57 smithi181 ceph-mon[29801]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:26:57.816 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:57 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:26:57.817 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:57 smithi181 ceph-mon[29801]: Added host smithi196 2024-05-03T20:26:57.817 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:57 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:26:57.817 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:57 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:26:58.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:58 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:26:58.961 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.959+0000 7f3689424700 1 -- 172.21.15.181:0/1752199343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36841030e0 msgr2=0x7f36841034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:58.961 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.959+0000 7f3689424700 1 --2- 172.21.15.181:0/1752199343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36841030e0 0x7f36841034c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3678009910 tx=0x7f3678009c20 comp rx=0 tx=0).stop 2024-05-03T20:26:58.962 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.960+0000 7f3689424700 1 -- 172.21.15.181:0/1752199343 shutdown_connections 2024-05-03T20:26:58.962 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.960+0000 7f3689424700 1 --2- 172.21.15.181:0/1752199343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36841030e0 0x7f36841034c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:58.962 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.960+0000 7f3689424700 1 -- 172.21.15.181:0/1752199343 >> 172.21.15.181:0/1752199343 conn(0x7f36840fe990 msgr2=0x7f3684100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:58.962 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.960+0000 7f3689424700 1 -- 172.21.15.181:0/1752199343 shutdown_connections 2024-05-03T20:26:58.962 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.960+0000 7f3689424700 1 -- 172.21.15.181:0/1752199343 wait complete. 2024-05-03T20:26:58.962 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.960+0000 7f3689424700 1 Processor -- start 2024-05-03T20:26:58.962 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.960+0000 7f3689424700 1 -- start start 2024-05-03T20:26:58.962 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.960+0000 7f3689424700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36841030e0 0x7f368419e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:58.962 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.961+0000 7f3689424700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f368410d360 con 0x7f36841030e0 2024-05-03T20:26:58.962 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.961+0000 7f3682ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36841030e0 0x7f368419e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:58.962 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.961+0000 7f3682ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36841030e0 0x7f368419e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48790/0 (socket says 172.21.15.181:48790) 2024-05-03T20:26:58.963 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.961+0000 7f3682ffd700 1 -- 172.21.15.181:0/3304904385 learned_addr learned my addr 172.21.15.181:0/3304904385 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:26:58.963 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.961+0000 7f3682ffd700 1 -- 172.21.15.181:0/3304904385 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368419eb90 con 0x7f36841030e0 2024-05-03T20:26:58.963 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.961+0000 7f3682ffd700 1 --2- 172.21.15.181:0/3304904385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36841030e0 0x7f368419e650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f36780056d0 tx=0x7f36780057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:58.963 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.961+0000 7f366bfff700 1 -- 172.21.15.181:0/3304904385 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3678026070 con 0x7f36841030e0 2024-05-03T20:26:58.963 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.961+0000 7f366bfff700 1 -- 172.21.15.181:0/3304904385 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f367800bba0 con 0x7f36841030e0 2024-05-03T20:26:58.963 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.961+0000 7f366bfff700 1 -- 172.21.15.181:0/3304904385 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3678003920 con 0x7f36841030e0 2024-05-03T20:26:58.963 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.961+0000 7f3689424700 1 -- 172.21.15.181:0/3304904385 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f368419ee80 con 0x7f36841030e0 2024-05-03T20:26:58.963 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.961+0000 7f3689424700 1 -- 172.21.15.181:0/3304904385 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36841988f0 con 0x7f36841030e0 2024-05-03T20:26:58.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.962+0000 7f3689424700 1 -- 172.21.15.181:0/3304904385 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36841045c0 con 0x7f36841030e0 2024-05-03T20:26:58.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.962+0000 7f366bfff700 1 -- 172.21.15.181:0/3304904385 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f3678029070 con 0x7f36841030e0 2024-05-03T20:26:58.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.963+0000 7f366bfff700 1 --2- 172.21.15.181:0/3304904385 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f366c03e510 0x7f366c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:26:58.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.963+0000 7f366bfff700 1 -- 172.21.15.181:0/3304904385 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f367801e070 con 0x7f36841030e0 2024-05-03T20:26:58.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.963+0000 7f36827fc700 1 --2- 172.21.15.181:0/3304904385 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f366c03e510 0x7f366c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:26:58.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.963+0000 7f36827fc700 1 --2- 172.21.15.181:0/3304904385 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f366c03e510 0x7f366c0409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3674006fd0 tx=0x7f3674006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:26:58.968 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:58.968+0000 7f366bfff700 1 -- 172.21.15.181:0/3304904385 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3678013070 con 0x7f36841030e0 2024-05-03T20:26:59.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:59.191+0000 7f3689424700 1 -- 172.21.15.181:0/3304904385 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3684199cc0 con 0x7f366c03e510 2024-05-03T20:26:59.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:59.193+0000 7f366bfff700 1 -- 172.21.15.181:0/3304904385 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f3684199cc0 con 0x7f366c03e510 2024-05-03T20:26:59.193 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:26:59.193 INFO:teuthology.orchestra.run.smithi181.stdout:[{"addr": "172.21.15.181", "hostname": "smithi181", "labels": [], "status": ""}, {"addr": "172.21.15.196", "hostname": "smithi196", "labels": [], "status": ""}] 2024-05-03T20:26:59.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:59.195+0000 7f3689424700 1 -- 172.21.15.181:0/3304904385 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f366c03e510 msgr2=0x7f366c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:59.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:59.195+0000 7f3689424700 1 --2- 172.21.15.181:0/3304904385 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f366c03e510 0x7f366c0409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3674006fd0 tx=0x7f3674006e40 comp rx=0 tx=0).stop 2024-05-03T20:26:59.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:59.195+0000 7f3689424700 1 -- 172.21.15.181:0/3304904385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36841030e0 msgr2=0x7f368419e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:26:59.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:59.195+0000 7f3689424700 1 --2- 172.21.15.181:0/3304904385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36841030e0 0x7f368419e650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f36780056d0 tx=0x7f36780057b0 comp rx=0 tx=0).stop 2024-05-03T20:26:59.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:59.195+0000 7f3689424700 1 -- 172.21.15.181:0/3304904385 shutdown_connections 2024-05-03T20:26:59.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:59.195+0000 7f3689424700 1 --2- 172.21.15.181:0/3304904385 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f366c03e510 0x7f366c0409d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:59.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:59.195+0000 7f3689424700 1 --2- 172.21.15.181:0/3304904385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36841030e0 0x7f368419e650 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:26:59.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:59.195+0000 7f3689424700 1 -- 172.21.15.181:0/3304904385 >> 172.21.15.181:0/3304904385 conn(0x7f36840fe990 msgr2=0x7f3684104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:26:59.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:59.195+0000 7f3689424700 1 -- 172.21.15.181:0/3304904385 shutdown_connections 2024-05-03T20:26:59.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:26:59.195+0000 7f3689424700 1 -- 172.21.15.181:0/3304904385 wait complete. 2024-05-03T20:26:59.759 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:59 smithi181 ceph-mon[29801]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:26:59.759 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:26:59 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:26:59.787 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi204 2024-05-03T20:26:59.788 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-03T20:26:59.788 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.conf 2024-05-03T20:26:59.807 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-03T20:26:59.807 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-03T20:26:59.864 INFO:tasks.cephadm:Adding host smithi204 to orchestrator... 2024-05-03T20:26:59.864 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph orch host add smithi204 2024-05-03T20:27:00.112 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:27:00.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:00 smithi181 ceph-mon[29801]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T20:27:01.638 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.636+0000 7f6e97e2b700 1 -- 172.21.15.181:0/2321024552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e901016f0 msgr2=0x7f6e90101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:01.638 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.636+0000 7f6e97e2b700 1 --2- 172.21.15.181:0/2321024552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e901016f0 0x7f6e90101ad0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000b0a0 tx=0x7f6e8000b3b0 comp rx=0 tx=0).stop 2024-05-03T20:27:01.638 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.636+0000 7f6e97e2b700 1 -- 172.21.15.181:0/2321024552 shutdown_connections 2024-05-03T20:27:01.638 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.636+0000 7f6e97e2b700 1 --2- 172.21.15.181:0/2321024552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e901016f0 0x7f6e90101ad0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:01.638 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.636+0000 7f6e97e2b700 1 -- 172.21.15.181:0/2321024552 >> 172.21.15.181:0/2321024552 conn(0x7f6e90078490 msgr2=0x7f6e900788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:27:01.638 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.636+0000 7f6e97e2b700 1 -- 172.21.15.181:0/2321024552 shutdown_connections 2024-05-03T20:27:01.638 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.636+0000 7f6e97e2b700 1 -- 172.21.15.181:0/2321024552 wait complete. 2024-05-03T20:27:01.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.637+0000 7f6e97e2b700 1 Processor -- start 2024-05-03T20:27:01.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.637+0000 7f6e97e2b700 1 -- start start 2024-05-03T20:27:01.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.637+0000 7f6e97e2b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e901016f0 0x7f6e9019e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:01.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.637+0000 7f6e97e2b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9010d350 con 0x7f6e901016f0 2024-05-03T20:27:01.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.637+0000 7f6e95bc7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e901016f0 0x7f6e9019e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:01.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.637+0000 7f6e95bc7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e901016f0 0x7f6e9019e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48800/0 (socket says 172.21.15.181:48800) 2024-05-03T20:27:01.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.637+0000 7f6e95bc7700 1 -- 172.21.15.181:0/268186595 learned_addr learned my addr 172.21.15.181:0/268186595 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:27:01.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.638+0000 7f6e95bc7700 1 -- 172.21.15.181:0/268186595 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e9019eab0 con 0x7f6e901016f0 2024-05-03T20:27:01.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.638+0000 7f6e95bc7700 1 --2- 172.21.15.181:0/268186595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e901016f0 0x7f6e9019e570 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6e80006130 tx=0x7f6e800097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:27:01.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.638+0000 7f6e8effd700 1 -- 172.21.15.181:0/268186595 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e80021020 con 0x7f6e901016f0 2024-05-03T20:27:01.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.638+0000 7f6e8effd700 1 -- 172.21.15.181:0/268186595 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e800079b0 con 0x7f6e901016f0 2024-05-03T20:27:01.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.638+0000 7f6e97e2b700 1 -- 172.21.15.181:0/268186595 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e9019ed40 con 0x7f6e901016f0 2024-05-03T20:27:01.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.638+0000 7f6e8effd700 1 -- 172.21.15.181:0/268186595 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e80017040 con 0x7f6e901016f0 2024-05-03T20:27:01.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.638+0000 7f6e97e2b700 1 -- 172.21.15.181:0/268186595 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e90198640 con 0x7f6e901016f0 2024-05-03T20:27:01.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.639+0000 7f6e8effd700 1 -- 172.21.15.181:0/268186595 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f6e80024020 con 0x7f6e901016f0 2024-05-03T20:27:01.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.639+0000 7f6e97e2b700 1 -- 172.21.15.181:0/268186595 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e901047b0 con 0x7f6e901016f0 2024-05-03T20:27:01.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.639+0000 7f6e8effd700 1 --2- 172.21.15.181:0/268186595 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f6e7803e510 0x7f6e780409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:01.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.639+0000 7f6e8effd700 1 -- 172.21.15.181:0/268186595 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6e8000f050 con 0x7f6e901016f0 2024-05-03T20:27:01.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.639+0000 7f6e953c6700 1 --2- 172.21.15.181:0/268186595 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f6e7803e510 0x7f6e780409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:01.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.639+0000 7f6e953c6700 1 --2- 172.21.15.181:0/268186595 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f6e7803e510 0x7f6e780409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6e84006fd0 tx=0x7f6e84006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:27:01.644 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.644+0000 7f6e8effd700 1 -- 172.21.15.181:0/268186595 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6e80057030 con 0x7f6e901016f0 2024-05-03T20:27:01.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:01 smithi181 ceph-mon[29801]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:01.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:01.854+0000 7f6e97e2b700 1 -- 172.21.15.181:0/268186595 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}) -- 0x7f6e90199b70 con 0x7f6e7803e510 2024-05-03T20:27:03.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:03 smithi181 ceph-mon[29801]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:03.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:03 smithi181 ceph-mon[29801]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:27:03.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:03 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:03.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:03 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:03.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:03 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:03.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:03 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:03.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:03 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:27:03.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:03 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:03.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:03 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:27:03.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:03 smithi181 ceph-mon[29801]: Updating smithi196:/etc/ceph/ceph.conf 2024-05-03T20:27:03.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:03 smithi181 ceph-mon[29801]: Updating smithi196:/var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T20:27:03.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:03 smithi181 ceph-mon[29801]: Deploying cephadm binary to smithi204 2024-05-03T20:27:03.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:03 smithi181 ceph-mon[29801]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-05-03T20:27:04.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:04 smithi181 ceph-mon[29801]: Updating smithi196:/var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T20:27:05.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:05 smithi181 ceph-mon[29801]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:05.929 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:05.929+0000 7f6e8effd700 1 -- 172.21.15.181:0/268186595 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f6e90199b70 con 0x7f6e7803e510 2024-05-03T20:27:05.930 INFO:teuthology.orchestra.run.smithi181.stdout:Added host 'smithi204' with addr '172.21.15.204' 2024-05-03T20:27:05.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:05.932+0000 7f6e97e2b700 1 -- 172.21.15.181:0/268186595 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f6e7803e510 msgr2=0x7f6e780409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:05.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:05.932+0000 7f6e97e2b700 1 --2- 172.21.15.181:0/268186595 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f6e7803e510 0x7f6e780409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6e84006fd0 tx=0x7f6e84006e40 comp rx=0 tx=0).stop 2024-05-03T20:27:05.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:05.932+0000 7f6e97e2b700 1 -- 172.21.15.181:0/268186595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e901016f0 msgr2=0x7f6e9019e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:05.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:05.932+0000 7f6e97e2b700 1 --2- 172.21.15.181:0/268186595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e901016f0 0x7f6e9019e570 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6e80006130 tx=0x7f6e800097d0 comp rx=0 tx=0).stop 2024-05-03T20:27:05.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:05.932+0000 7f6e97e2b700 1 -- 172.21.15.181:0/268186595 shutdown_connections 2024-05-03T20:27:05.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:05.932+0000 7f6e97e2b700 1 --2- 172.21.15.181:0/268186595 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f6e7803e510 0x7f6e780409d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:05.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:05.932+0000 7f6e97e2b700 1 --2- 172.21.15.181:0/268186595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e901016f0 0x7f6e9019e570 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:05.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:05.932+0000 7f6e97e2b700 1 -- 172.21.15.181:0/268186595 >> 172.21.15.181:0/268186595 conn(0x7f6e90078490 msgr2=0x7f6e90104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:27:05.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:05.932+0000 7f6e97e2b700 1 -- 172.21.15.181:0/268186595 shutdown_connections 2024-05-03T20:27:05.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:05.932+0000 7f6e97e2b700 1 -- 172.21.15.181:0/268186595 wait complete. 2024-05-03T20:27:06.555 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:06 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:06.555 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:06 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:06.555 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:06 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:06.555 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:06 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:06.555 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:06 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:27:06.581 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-03T20:27:06.814 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:27:07.777 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:07 smithi181 ceph-mon[29801]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:07.777 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:07 smithi181 ceph-mon[29801]: Added host smithi204 2024-05-03T20:27:07.777 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:07.778 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:08.258 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.256+0000 7f65b2f0e700 1 -- 172.21.15.181:0/1068128267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65ac101fe0 msgr2=0x7f65ac1023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:08.258 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.256+0000 7f65b2f0e700 1 --2- 172.21.15.181:0/1068128267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65ac101fe0 0x7f65ac1023c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009910 tx=0x7f65a0009c20 comp rx=0 tx=0).stop 2024-05-03T20:27:08.258 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.256+0000 7f65b2f0e700 1 -- 172.21.15.181:0/1068128267 shutdown_connections 2024-05-03T20:27:08.258 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.256+0000 7f65b2f0e700 1 --2- 172.21.15.181:0/1068128267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65ac101fe0 0x7f65ac1023c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:08.259 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.256+0000 7f65b2f0e700 1 -- 172.21.15.181:0/1068128267 >> 172.21.15.181:0/1068128267 conn(0x7f65ac0fd870 msgr2=0x7f65ac0ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:27:08.259 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.256+0000 7f65b2f0e700 1 -- 172.21.15.181:0/1068128267 shutdown_connections 2024-05-03T20:27:08.259 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.256+0000 7f65b2f0e700 1 -- 172.21.15.181:0/1068128267 wait complete. 2024-05-03T20:27:08.259 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.257+0000 7f65b2f0e700 1 Processor -- start 2024-05-03T20:27:08.259 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.257+0000 7f65b2f0e700 1 -- start start 2024-05-03T20:27:08.259 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.257+0000 7f65b2f0e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65ac101fe0 0x7f65ac19d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:08.259 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.257+0000 7f65b2f0e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac10c260 con 0x7f65ac101fe0 2024-05-03T20:27:08.259 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.258+0000 7f65b0caa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65ac101fe0 0x7f65ac19d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:08.259 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.258+0000 7f65b0caa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65ac101fe0 0x7f65ac19d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51508/0 (socket says 172.21.15.181:51508) 2024-05-03T20:27:08.259 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.258+0000 7f65b0caa700 1 -- 172.21.15.181:0/746468944 learned_addr learned my addr 172.21.15.181:0/746468944 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:27:08.259 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.258+0000 7f65b0caa700 1 -- 172.21.15.181:0/746468944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ac19dad0 con 0x7f65ac101fe0 2024-05-03T20:27:08.259 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.258+0000 7f65b0caa700 1 --2- 172.21.15.181:0/746468944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65ac101fe0 0x7f65ac19d590 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f65a00056d0 tx=0x7f65a00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:27:08.259 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.258+0000 7f65a9ffb700 1 -- 172.21.15.181:0/746468944 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65a0026070 con 0x7f65ac101fe0 2024-05-03T20:27:08.259 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.258+0000 7f65a9ffb700 1 -- 172.21.15.181:0/746468944 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65a000bba0 con 0x7f65ac101fe0 2024-05-03T20:27:08.260 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.258+0000 7f65b2f0e700 1 -- 172.21.15.181:0/746468944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65ac19ddc0 con 0x7f65ac101fe0 2024-05-03T20:27:08.260 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.258+0000 7f65a9ffb700 1 -- 172.21.15.181:0/746468944 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65a0003920 con 0x7f65ac101fe0 2024-05-03T20:27:08.260 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.258+0000 7f65b2f0e700 1 -- 172.21.15.181:0/746468944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65ac197830 con 0x7f65ac101fe0 2024-05-03T20:27:08.260 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.259+0000 7f65a9ffb700 1 -- 172.21.15.181:0/746468944 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f65a0029070 con 0x7f65ac101fe0 2024-05-03T20:27:08.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.259+0000 7f65b2f0e700 1 -- 172.21.15.181:0/746468944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65ac1034c0 con 0x7f65ac101fe0 2024-05-03T20:27:08.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.259+0000 7f65a9ffb700 1 --2- 172.21.15.181:0/746468944 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f659c03e510 0x7f659c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:08.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.259+0000 7f65a9ffb700 1 -- 172.21.15.181:0/746468944 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f65a001e070 con 0x7f65ac101fe0 2024-05-03T20:27:08.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.260+0000 7f65abfff700 1 --2- 172.21.15.181:0/746468944 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f659c03e510 0x7f659c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:08.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.260+0000 7f65abfff700 1 --2- 172.21.15.181:0/746468944 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f659c03e510 0x7f659c0409d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6598006fd0 tx=0x7f6598006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:27:08.265 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.265+0000 7f65a9ffb700 1 -- 172.21.15.181:0/746468944 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f65a0013070 con 0x7f65ac101fe0 2024-05-03T20:27:08.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.471+0000 7f65b2f0e700 1 -- 172.21.15.181:0/746468944 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f65ac198b90 con 0x7f659c03e510 2024-05-03T20:27:08.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.472+0000 7f65a9ffb700 1 -- 172.21.15.181:0/746468944 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f65ac198b90 con 0x7f659c03e510 2024-05-03T20:27:08.473 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:27:08.473 INFO:teuthology.orchestra.run.smithi181.stdout:[{"addr": "172.21.15.181", "hostname": "smithi181", "labels": [], "status": ""}, {"addr": "172.21.15.196", "hostname": "smithi196", "labels": [], "status": ""}, {"addr": "172.21.15.204", "hostname": "smithi204", "labels": [], "status": ""}] 2024-05-03T20:27:08.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.475+0000 7f65b2f0e700 1 -- 172.21.15.181:0/746468944 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f659c03e510 msgr2=0x7f659c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:08.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.475+0000 7f65b2f0e700 1 --2- 172.21.15.181:0/746468944 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f659c03e510 0x7f659c0409d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6598006fd0 tx=0x7f6598006e40 comp rx=0 tx=0).stop 2024-05-03T20:27:08.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.475+0000 7f65b2f0e700 1 -- 172.21.15.181:0/746468944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65ac101fe0 msgr2=0x7f65ac19d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:08.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.475+0000 7f65b2f0e700 1 --2- 172.21.15.181:0/746468944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65ac101fe0 0x7f65ac19d590 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f65a00056d0 tx=0x7f65a00057b0 comp rx=0 tx=0).stop 2024-05-03T20:27:08.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.475+0000 7f65b2f0e700 1 -- 172.21.15.181:0/746468944 shutdown_connections 2024-05-03T20:27:08.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.475+0000 7f65b2f0e700 1 --2- 172.21.15.181:0/746468944 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f659c03e510 0x7f659c0409d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:08.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.475+0000 7f65b2f0e700 1 --2- 172.21.15.181:0/746468944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65ac101fe0 0x7f65ac19d590 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:08.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.475+0000 7f65b2f0e700 1 -- 172.21.15.181:0/746468944 >> 172.21.15.181:0/746468944 conn(0x7f65ac0fd870 msgr2=0x7f65ac109b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:27:08.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.475+0000 7f65b2f0e700 1 -- 172.21.15.181:0/746468944 shutdown_connections 2024-05-03T20:27:08.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:08.475+0000 7f65b2f0e700 1 -- 172.21.15.181:0/746468944 wait complete. 2024-05-03T20:27:09.078 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-03T20:27:09.078 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-03T20:27:09.310 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:27:09.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:09 smithi181 ceph-mon[29801]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:09.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:09 smithi181 ceph-mon[29801]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T20:27:09.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:09 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:10.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.736+0000 7fef583a1700 1 -- 172.21.15.181:0/596690499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef50101fe0 msgr2=0x7fef501023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:10.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.736+0000 7fef583a1700 1 --2- 172.21.15.181:0/596690499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef50101fe0 0x7fef501023c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fef40009910 tx=0x7fef40009c20 comp rx=0 tx=0).stop 2024-05-03T20:27:10.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.736+0000 7fef583a1700 1 -- 172.21.15.181:0/596690499 shutdown_connections 2024-05-03T20:27:10.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.736+0000 7fef583a1700 1 --2- 172.21.15.181:0/596690499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef50101fe0 0x7fef501023c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:10.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.736+0000 7fef583a1700 1 -- 172.21.15.181:0/596690499 >> 172.21.15.181:0/596690499 conn(0x7fef50078a70 msgr2=0x7fef500ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:27:10.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.736+0000 7fef583a1700 1 -- 172.21.15.181:0/596690499 shutdown_connections 2024-05-03T20:27:10.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.736+0000 7fef583a1700 1 -- 172.21.15.181:0/596690499 wait complete. 2024-05-03T20:27:10.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.737+0000 7fef583a1700 1 Processor -- start 2024-05-03T20:27:10.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.737+0000 7fef583a1700 1 -- start start 2024-05-03T20:27:10.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.737+0000 7fef583a1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef50101fe0 0x7fef5019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:10.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.737+0000 7fef583a1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fef5010c260 con 0x7fef50101fe0 2024-05-03T20:27:10.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.737+0000 7fef5613d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef50101fe0 0x7fef5019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:10.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.737+0000 7fef5613d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef50101fe0 0x7fef5019e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51520/0 (socket says 172.21.15.181:51520) 2024-05-03T20:27:10.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.737+0000 7fef5613d700 1 -- 172.21.15.181:0/4181499110 learned_addr learned my addr 172.21.15.181:0/4181499110 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:27:10.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.738+0000 7fef5613d700 1 -- 172.21.15.181:0/4181499110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef5019eb60 con 0x7fef50101fe0 2024-05-03T20:27:10.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.738+0000 7fef5613d700 1 --2- 172.21.15.181:0/4181499110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef50101fe0 0x7fef5019e620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fef40006010 tx=0x7fef40005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:27:10.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.738+0000 7fef477fe700 1 -- 172.21.15.181:0/4181499110 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef40026070 con 0x7fef50101fe0 2024-05-03T20:27:10.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.738+0000 7fef477fe700 1 -- 172.21.15.181:0/4181499110 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef4000bba0 con 0x7fef50101fe0 2024-05-03T20:27:10.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.738+0000 7fef583a1700 1 -- 172.21.15.181:0/4181499110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef5019ee50 con 0x7fef50101fe0 2024-05-03T20:27:10.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.738+0000 7fef477fe700 1 -- 172.21.15.181:0/4181499110 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef40003910 con 0x7fef50101fe0 2024-05-03T20:27:10.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.738+0000 7fef583a1700 1 -- 172.21.15.181:0/4181499110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef501988c0 con 0x7fef50101fe0 2024-05-03T20:27:10.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.739+0000 7fef477fe700 1 -- 172.21.15.181:0/4181499110 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fef40029070 con 0x7fef50101fe0 2024-05-03T20:27:10.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.739+0000 7fef583a1700 1 -- 172.21.15.181:0/4181499110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef501034c0 con 0x7fef50101fe0 2024-05-03T20:27:10.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.739+0000 7fef477fe700 1 --2- 172.21.15.181:0/4181499110 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fef3c03e570 0x7fef3c040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:10.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.739+0000 7fef477fe700 1 -- 172.21.15.181:0/4181499110 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fef4001e070 con 0x7fef50101fe0 2024-05-03T20:27:10.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.739+0000 7fef5593c700 1 --2- 172.21.15.181:0/4181499110 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fef3c03e570 0x7fef3c040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:10.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.740+0000 7fef5593c700 1 --2- 172.21.15.181:0/4181499110 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fef3c03e570 0x7fef3c040a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fef4c006fd0 tx=0x7fef4c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:27:10.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.744+0000 7fef477fe700 1 -- 172.21.15.181:0/4181499110 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fef40013070 con 0x7fef50101fe0 2024-05-03T20:27:10.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:10.955+0000 7fef583a1700 1 -- 172.21.15.181:0/4181499110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fef500008d0 con 0x7fef50101fe0 2024-05-03T20:27:11.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:11.543+0000 7fef477fe700 1 -- 172.21.15.181:0/4181499110 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fef4000b6f0 con 0x7fef50101fe0 2024-05-03T20:27:11.546 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:11.546+0000 7fef583a1700 1 -- 172.21.15.181:0/4181499110 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fef3c03e570 msgr2=0x7fef3c040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:11.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:11.546+0000 7fef583a1700 1 --2- 172.21.15.181:0/4181499110 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fef3c03e570 0x7fef3c040a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fef4c006fd0 tx=0x7fef4c006e40 comp rx=0 tx=0).stop 2024-05-03T20:27:11.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:11.546+0000 7fef583a1700 1 -- 172.21.15.181:0/4181499110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef50101fe0 msgr2=0x7fef5019e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:11.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:11.546+0000 7fef583a1700 1 --2- 172.21.15.181:0/4181499110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef50101fe0 0x7fef5019e620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fef40006010 tx=0x7fef40005780 comp rx=0 tx=0).stop 2024-05-03T20:27:11.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:11.546+0000 7fef583a1700 1 -- 172.21.15.181:0/4181499110 shutdown_connections 2024-05-03T20:27:11.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:11.546+0000 7fef583a1700 1 --2- 172.21.15.181:0/4181499110 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fef3c03e570 0x7fef3c040a30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:11.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:11.546+0000 7fef583a1700 1 --2- 172.21.15.181:0/4181499110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef50101fe0 0x7fef5019e620 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:11.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:11.546+0000 7fef583a1700 1 -- 172.21.15.181:0/4181499110 >> 172.21.15.181:0/4181499110 conn(0x7fef50078a70 msgr2=0x7fef5006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:27:11.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:11.547+0000 7fef583a1700 1 -- 172.21.15.181:0/4181499110 shutdown_connections 2024-05-03T20:27:11.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:11.547+0000 7fef583a1700 1 -- 172.21.15.181:0/4181499110 wait complete. 2024-05-03T20:27:11.547 INFO:teuthology.orchestra.run.smithi181.stderr:adjusted tunables profile to default 2024-05-03T20:27:11.904 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:11 smithi181 ceph-mon[29801]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:11.904 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:11 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/4181499110' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-03T20:27:12.233 INFO:tasks.cephadm:Adding mon.a on smithi181 2024-05-03T20:27:12.233 INFO:tasks.cephadm:Adding mon.b on smithi196 2024-05-03T20:27:12.233 INFO:tasks.cephadm:Adding mon.c on smithi204 2024-05-03T20:27:12.234 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph orch apply mon '3;smithi181:172.21.15.181=a;smithi196:172.21.15.196=b;smithi204:172.21.15.204=c' 2024-05-03T20:27:12.473 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T20:27:12.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:12 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/4181499110' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-03T20:27:12.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:12 smithi181 ceph-mon[29801]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T20:27:12.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:12.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:12.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:12.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:12.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:27:12.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:12.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:27:12.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:12 smithi181 ceph-mon[29801]: Updating smithi204:/etc/ceph/ceph.conf 2024-05-03T20:27:12.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:12.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:12.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:13.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.936+0000 7faf81228700 1 -- 172.21.15.204:0/3140374982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf7c1016f0 msgr2=0x7faf7c101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:13.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.936+0000 7faf81228700 1 --2- 172.21.15.204:0/3140374982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf7c1016f0 0x7faf7c101ad0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7faf70009910 tx=0x7faf70009c20 comp rx=0 tx=0).stop 2024-05-03T20:27:13.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.936+0000 7faf81228700 1 -- 172.21.15.204:0/3140374982 shutdown_connections 2024-05-03T20:27:13.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.936+0000 7faf81228700 1 --2- 172.21.15.204:0/3140374982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf7c1016f0 0x7faf7c101ad0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:13.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.936+0000 7faf81228700 1 -- 172.21.15.204:0/3140374982 >> 172.21.15.204:0/3140374982 conn(0x7faf7c078490 msgr2=0x7faf7c0788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:27:13.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.936+0000 7faf81228700 1 -- 172.21.15.204:0/3140374982 shutdown_connections 2024-05-03T20:27:13.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.936+0000 7faf81228700 1 -- 172.21.15.204:0/3140374982 wait complete. 2024-05-03T20:27:13.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.937+0000 7faf81228700 1 Processor -- start 2024-05-03T20:27:13.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.937+0000 7faf81228700 1 -- start start 2024-05-03T20:27:13.938 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.937+0000 7faf81228700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf7c1016f0 0x7faf7c19e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:13.938 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.937+0000 7faf81228700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c10d380 con 0x7faf7c1016f0 2024-05-03T20:27:13.938 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.938+0000 7faf7ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf7c1016f0 0x7faf7c19e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:13.939 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.938+0000 7faf7ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf7c1016f0 0x7faf7c19e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.204:48984/0 (socket says 172.21.15.204:48984) 2024-05-03T20:27:13.939 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.938+0000 7faf7ad9d700 1 -- 172.21.15.204:0/2728932611 learned_addr learned my addr 172.21.15.204:0/2728932611 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-03T20:27:13.939 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.938+0000 7faf7ad9d700 1 -- 172.21.15.204:0/2728932611 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7c19ebe0 con 0x7faf7c1016f0 2024-05-03T20:27:13.939 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.938+0000 7faf7ad9d700 1 --2- 172.21.15.204:0/2728932611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf7c1016f0 0x7faf7c19e6a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faf700056d0 tx=0x7faf700057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:27:13.939 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.938+0000 7faf63fff700 1 -- 172.21.15.204:0/2728932611 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf70026070 con 0x7faf7c1016f0 2024-05-03T20:27:13.939 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.938+0000 7faf63fff700 1 -- 172.21.15.204:0/2728932611 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf7000bba0 con 0x7faf7c1016f0 2024-05-03T20:27:13.939 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.938+0000 7faf81228700 1 -- 172.21.15.204:0/2728932611 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf7c19ee70 con 0x7faf7c1016f0 2024-05-03T20:27:13.939 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.938+0000 7faf63fff700 1 -- 172.21.15.204:0/2728932611 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf70003920 con 0x7faf7c1016f0 2024-05-03T20:27:13.939 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.939+0000 7faf81228700 1 -- 172.21.15.204:0/2728932611 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf7c198800 con 0x7faf7c1016f0 2024-05-03T20:27:13.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.939+0000 7faf81228700 1 -- 172.21.15.204:0/2728932611 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf7c1047b0 con 0x7faf7c1016f0 2024-05-03T20:27:13.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.940+0000 7faf63fff700 1 -- 172.21.15.204:0/2728932611 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7faf70029070 con 0x7faf7c1016f0 2024-05-03T20:27:13.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.940+0000 7faf63fff700 1 --2- 172.21.15.204:0/2728932611 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7faf6403e510 0x7faf640409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:13.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.940+0000 7faf63fff700 1 -- 172.21.15.204:0/2728932611 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7faf7001e070 con 0x7faf7c1016f0 2024-05-03T20:27:13.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.940+0000 7faf7a59c700 1 --2- 172.21.15.204:0/2728932611 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7faf6403e510 0x7faf640409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:13.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.941+0000 7faf7a59c700 1 --2- 172.21.15.204:0/2728932611 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7faf6403e510 0x7faf640409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faf6c006fd0 tx=0x7faf6c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:27:13.945 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:13.945+0000 7faf63fff700 1 -- 172.21.15.204:0/2728932611 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7faf70013070 con 0x7faf7c1016f0 2024-05-03T20:27:13.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:13 smithi181 ceph-mon[29801]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:13.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:13 smithi181 ceph-mon[29801]: Updating smithi204:/var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T20:27:13.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:13 smithi181 ceph-mon[29801]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-05-03T20:27:13.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:13 smithi181 ceph-mon[29801]: Updating smithi204:/var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T20:27:14.157 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:14.157+0000 7faf81228700 1 -- 172.21.15.204:0/2728932611 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi181:172.21.15.181=a;smithi196:172.21.15.196=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}) -- 0x7faf7c199d10 con 0x7faf6403e510 2024-05-03T20:27:14.163 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:14.164+0000 7faf63fff700 1 -- 172.21.15.204:0/2728932611 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7faf7c199d10 con 0x7faf6403e510 2024-05-03T20:27:14.164 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mon update... 2024-05-03T20:27:14.166 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:14.166+0000 7faf81228700 1 -- 172.21.15.204:0/2728932611 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7faf6403e510 msgr2=0x7faf640409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:14.166 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:14.166+0000 7faf81228700 1 --2- 172.21.15.204:0/2728932611 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7faf6403e510 0x7faf640409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faf6c006fd0 tx=0x7faf6c006e40 comp rx=0 tx=0).stop 2024-05-03T20:27:14.166 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:14.166+0000 7faf81228700 1 -- 172.21.15.204:0/2728932611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf7c1016f0 msgr2=0x7faf7c19e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:14.166 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:14.166+0000 7faf81228700 1 --2- 172.21.15.204:0/2728932611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf7c1016f0 0x7faf7c19e6a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faf700056d0 tx=0x7faf700057b0 comp rx=0 tx=0).stop 2024-05-03T20:27:14.166 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:14.166+0000 7faf81228700 1 -- 172.21.15.204:0/2728932611 shutdown_connections 2024-05-03T20:27:14.166 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:14.166+0000 7faf81228700 1 --2- 172.21.15.204:0/2728932611 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7faf6403e510 0x7faf640409d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:14.166 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:14.166+0000 7faf81228700 1 --2- 172.21.15.204:0/2728932611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf7c1016f0 0x7faf7c19e6a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:14.166 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:14.166+0000 7faf81228700 1 -- 172.21.15.204:0/2728932611 >> 172.21.15.204:0/2728932611 conn(0x7faf7c078490 msgr2=0x7faf7c104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:27:14.166 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:14.166+0000 7faf81228700 1 -- 172.21.15.204:0/2728932611 shutdown_connections 2024-05-03T20:27:14.167 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:14.166+0000 7faf81228700 1 -- 172.21.15.204:0/2728932611 wait complete. 2024-05-03T20:27:14.743 DEBUG:teuthology.orchestra.run.smithi196:mon.b> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mon.b.service 2024-05-03T20:27:14.745 DEBUG:teuthology.orchestra.run.smithi204:mon.c> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mon.c.service 2024-05-03T20:27:14.747 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-03T20:27:14.748 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-03T20:27:15.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:15 smithi181 ceph-mon[29801]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:15.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:15 smithi181 ceph-mon[29801]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi181:172.21.15.181=a;smithi196:172.21.15.196=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:27:15.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:15 smithi181 ceph-mon[29801]: Saving service mon spec with placement smithi181:172.21.15.181=a;smithi196:172.21.15.196=b;smithi204:172.21.15.204=c;count:3 2024-05-03T20:27:15.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:15 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:15.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:15 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:27:15.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:15 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:15.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:15 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:27:15.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:15 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:15.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:15 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T20:27:15.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:15 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:15.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:15 smithi181 ceph-mon[29801]: Deploying daemon mon.c on smithi204 2024-05-03T20:27:15.519 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T20:27:17.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:17 smithi181 ceph-mon[29801]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:18.760 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.760+0000 7f3a1cd6a700 1 -- 172.21.15.204:0/1161814608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a18105300 msgr2=0x7f3a181056e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:18.760 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.760+0000 7f3a1cd6a700 1 --2- 172.21.15.204:0/1161814608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a18105300 0x7f3a181056e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009910 tx=0x7f3a08009c20 comp rx=0 tx=0).stop 2024-05-03T20:27:18.760 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.760+0000 7f3a1cd6a700 1 -- 172.21.15.204:0/1161814608 shutdown_connections 2024-05-03T20:27:18.760 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.760+0000 7f3a1cd6a700 1 --2- 172.21.15.204:0/1161814608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a18105300 0x7f3a181056e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:18.760 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.760+0000 7f3a1cd6a700 1 -- 172.21.15.204:0/1161814608 >> 172.21.15.204:0/1161814608 conn(0x7f3a18100b70 msgr2=0x7f3a18102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:27:18.760 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.760+0000 7f3a1cd6a700 1 -- 172.21.15.204:0/1161814608 shutdown_connections 2024-05-03T20:27:18.760 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.760+0000 7f3a1cd6a700 1 -- 172.21.15.204:0/1161814608 wait complete. 2024-05-03T20:27:18.762 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.761+0000 7f3a1cd6a700 1 Processor -- start 2024-05-03T20:27:18.762 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.761+0000 7f3a1cd6a700 1 -- start start 2024-05-03T20:27:18.762 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.761+0000 7f3a1cd6a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a18105300 0x7f3a181a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:18.762 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.761+0000 7f3a1cd6a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1810f580 con 0x7f3a18105300 2024-05-03T20:27:18.762 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.762+0000 7f3a1659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a18105300 0x7f3a181a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:18.762 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.762+0000 7f3a1659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a18105300 0x7f3a181a07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.204:50254/0 (socket says 172.21.15.204:50254) 2024-05-03T20:27:18.762 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.762+0000 7f3a1659c700 1 -- 172.21.15.204:0/3501517177 learned_addr learned my addr 172.21.15.204:0/3501517177 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-03T20:27:18.762 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.762+0000 7f3a1659c700 1 -- 172.21.15.204:0/3501517177 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a181a0ce0 con 0x7f3a18105300 2024-05-03T20:27:18.762 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.762+0000 7f3a1659c700 1 --2- 172.21.15.204:0/3501517177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a18105300 0x7f3a181a07a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3a080056d0 tx=0x7f3a080057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:27:18.762 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.762+0000 7f3a077fe700 1 -- 172.21.15.204:0/3501517177 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a08026070 con 0x7f3a18105300 2024-05-03T20:27:18.763 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.762+0000 7f3a077fe700 1 -- 172.21.15.204:0/3501517177 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a0800bba0 con 0x7f3a18105300 2024-05-03T20:27:18.763 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.762+0000 7f3a1cd6a700 1 -- 172.21.15.204:0/3501517177 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a181a0fd0 con 0x7f3a18105300 2024-05-03T20:27:18.763 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.762+0000 7f3a077fe700 1 -- 172.21.15.204:0/3501517177 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a08003920 con 0x7f3a18105300 2024-05-03T20:27:18.763 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.762+0000 7f3a1cd6a700 1 -- 172.21.15.204:0/3501517177 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a1819a960 con 0x7f3a18105300 2024-05-03T20:27:18.764 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.763+0000 7f3a1cd6a700 1 -- 172.21.15.204:0/3501517177 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a181067e0 con 0x7f3a18105300 2024-05-03T20:27:18.764 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.763+0000 7f3a077fe700 1 -- 172.21.15.204:0/3501517177 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f3a08029070 con 0x7f3a18105300 2024-05-03T20:27:18.764 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.763+0000 7f3a077fe700 1 --2- 172.21.15.204:0/3501517177 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3a0003e510 0x7f3a000409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:18.764 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.764+0000 7f3a077fe700 1 -- 172.21.15.204:0/3501517177 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3a0801e070 con 0x7f3a18105300 2024-05-03T20:27:18.764 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.764+0000 7f3a15d9b700 1 --2- 172.21.15.204:0/3501517177 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3a0003e510 0x7f3a000409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:18.764 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.764+0000 7f3a15d9b700 1 --2- 172.21.15.204:0/3501517177 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3a0003e510 0x7f3a000409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c006fd0 tx=0x7f3a0c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:27:18.768 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:18.769+0000 7f3a077fe700 1 -- 172.21.15.204:0/3501517177 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3a08013070 con 0x7f3a18105300 2024-05-03T20:27:19.074 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:19.073+0000 7f3a1cd6a700 1 -- 172.21.15.204:0/3501517177 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3a1819be90 con 0x7f3a18105300 2024-05-03T20:27:19.074 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:19.074+0000 7f3a077fe700 1 -- 172.21.15.204:0/3501517177 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f3a08030090 con 0x7f3a18105300 2024-05-03T20:27:19.074 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-03T20:27:19.074 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":1,"fsid":"1cd058ee-098b-11ef-bc97-c7b262605968","modified":"2024-05-03T20:25:36.511893Z","created":"2024-05-03T20:25:36.511893Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:3300","nonce":0},{"type":"v1","addr":"172.21.15.181:6789","nonce":0}]},"addr":"172.21.15.181:6789/0","public_addr":"172.21.15.181:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-03T20:27:19.076 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:19.076+0000 7f3a1cd6a700 1 -- 172.21.15.204:0/3501517177 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3a0003e510 msgr2=0x7f3a000409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:19.076 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:19.076+0000 7f3a1cd6a700 1 --2- 172.21.15.204:0/3501517177 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3a0003e510 0x7f3a000409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c006fd0 tx=0x7f3a0c006e40 comp rx=0 tx=0).stop 2024-05-03T20:27:19.076 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:19.076+0000 7f3a1cd6a700 1 -- 172.21.15.204:0/3501517177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a18105300 msgr2=0x7f3a181a07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:19.076 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:19.076+0000 7f3a1cd6a700 1 --2- 172.21.15.204:0/3501517177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a18105300 0x7f3a181a07a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3a080056d0 tx=0x7f3a080057b0 comp rx=0 tx=0).stop 2024-05-03T20:27:19.076 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:19.076+0000 7f3a1cd6a700 1 -- 172.21.15.204:0/3501517177 shutdown_connections 2024-05-03T20:27:19.077 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:19.076+0000 7f3a1cd6a700 1 --2- 172.21.15.204:0/3501517177 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3a0003e510 0x7f3a000409d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:19.077 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:19.076+0000 7f3a1cd6a700 1 --2- 172.21.15.204:0/3501517177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a18105300 0x7f3a181a07a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:19.077 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:19.076+0000 7f3a1cd6a700 1 -- 172.21.15.204:0/3501517177 >> 172.21.15.204:0/3501517177 conn(0x7f3a18100b70 msgr2=0x7f3a18106cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:27:19.077 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:19.077+0000 7f3a1cd6a700 1 -- 172.21.15.204:0/3501517177 shutdown_connections 2024-05-03T20:27:19.077 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:19.077+0000 7f3a1cd6a700 1 -- 172.21.15.204:0/3501517177 wait complete. 2024-05-03T20:27:19.077 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 1 2024-05-03T20:27:19.226 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 podman[33563]: 2024-05-03 20:27:19.202016597 +0000 UTC m=+0.365728916 container start f777a02cfbdfa59207ff2857b228cbc86f34e9b2db2897c0c514e9f535d1ee45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-c, ceph=True, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T20:27:19.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:19 smithi181 ceph-mon[29801]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:19.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:19 smithi181 ceph-mon[29801]: from='client.? 172.21.15.204:0/3501517177' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T20:27:19.566 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: set uid:gid to 167:167 (ceph:ceph) 2024-05-03T20:27:19.566 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev), process ceph-mon, pid 2 2024-05-03T20:27:19.566 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: pidfile_write: ignore empty --pid-file 2024-05-03T20:27:19.566 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: load: jerasure load: lrc 2024-05-03T20:27:19.566 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: RocksDB version: 7.9.2 2024-05-03T20:27:19.566 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Git sha 0 2024-05-03T20:27:19.566 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Compile date 2024-05-01 21:12:03 2024-05-03T20:27:19.566 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: DB SUMMARY 2024-05-03T20:27:19.567 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: DB Session ID: 12A0K61K6GW34ITONA1F 2024-05-03T20:27:19.567 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: CURRENT file: CURRENT 2024-05-03T20:27:19.567 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: IDENTITY file: IDENTITY 2024-05-03T20:27:19.567 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-03T20:27:19.567 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-03T20:27:19.567 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-03T20:27:19.567 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.error_if_exists: 0 2024-05-03T20:27:19.567 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.create_if_missing: 0 2024-05-03T20:27:19.567 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.paranoid_checks: 1 2024-05-03T20:27:19.567 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-03T20:27:19.567 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-03T20:27:19.567 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-03T20:27:19.567 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.env: 0x5565207c2cc0 2024-05-03T20:27:19.568 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.fs: PosixFileSystem 2024-05-03T20:27:19.568 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.info_log: 0x556522fb9da0 2024-05-03T20:27:19.568 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.max_file_opening_threads: 16 2024-05-03T20:27:19.568 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.statistics: (nil) 2024-05-03T20:27:19.568 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.use_fsync: 0 2024-05-03T20:27:19.568 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.max_log_file_size: 0 2024-05-03T20:27:19.568 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-03T20:27:19.568 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-03T20:27:19.568 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.keep_log_file_num: 1000 2024-05-03T20:27:19.568 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.recycle_log_file_num: 0 2024-05-03T20:27:19.568 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.allow_fallocate: 1 2024-05-03T20:27:19.568 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.allow_mmap_reads: 0 2024-05-03T20:27:19.568 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.allow_mmap_writes: 0 2024-05-03T20:27:19.568 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.use_direct_reads: 0 2024-05-03T20:27:19.568 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-03T20:27:19.569 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.create_missing_column_families: 0 2024-05-03T20:27:19.569 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.db_log_dir: 2024-05-03T20:27:19.569 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.wal_dir: 2024-05-03T20:27:19.569 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-03T20:27:19.570 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-03T20:27:19.570 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-03T20:27:19.571 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-03T20:27:19.571 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-03T20:27:19.571 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-03T20:27:19.571 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.advise_random_on_open: 1 2024-05-03T20:27:19.571 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.db_write_buffer_size: 0 2024-05-03T20:27:19.571 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.write_buffer_manager: 0x556522f8b0e0 2024-05-03T20:27:19.571 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-03T20:27:19.571 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-03T20:27:19.571 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-03T20:27:19.571 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.rate_limiter: (nil) 2024-05-03T20:27:19.571 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-03T20:27:19.571 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.wal_recovery_mode: 2 2024-05-03T20:27:19.571 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.enable_thread_tracking: 0 2024-05-03T20:27:19.571 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.enable_pipelined_write: 0 2024-05-03T20:27:19.572 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.unordered_write: 0 2024-05-03T20:27:19.572 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-03T20:27:19.572 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-03T20:27:19.572 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-03T20:27:19.572 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-03T20:27:19.572 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.row_cache: None 2024-05-03T20:27:19.572 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.wal_filter: None 2024-05-03T20:27:19.572 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-03T20:27:19.572 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.allow_ingest_behind: 0 2024-05-03T20:27:19.572 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.two_write_queues: 0 2024-05-03T20:27:19.572 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.manual_wal_flush: 0 2024-05-03T20:27:19.572 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.wal_compression: 0 2024-05-03T20:27:19.572 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.atomic_flush: 0 2024-05-03T20:27:19.572 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-03T20:27:19.572 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-03T20:27:19.573 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-03T20:27:19.573 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.log_readahead_size: 0 2024-05-03T20:27:19.573 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-03T20:27:19.573 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.best_efforts_recovery: 0 2024-05-03T20:27:19.573 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-03T20:27:19.573 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-03T20:27:19.573 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.allow_data_in_errors: 0 2024-05-03T20:27:19.573 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.db_host_id: __hostname__ 2024-05-03T20:27:19.573 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-03T20:27:19.573 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.max_background_jobs: 2 2024-05-03T20:27:19.573 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.max_background_compactions: -1 2024-05-03T20:27:19.573 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.max_subcompactions: 1 2024-05-03T20:27:19.573 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-03T20:27:19.573 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-03T20:27:19.573 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-03T20:27:19.574 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.max_total_wal_size: 0 2024-05-03T20:27:19.574 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-03T20:27:19.574 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-03T20:27:19.574 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-03T20:27:19.574 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-03T20:27:19.574 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.max_open_files: -1 2024-05-03T20:27:19.574 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.bytes_per_sync: 0 2024-05-03T20:27:19.574 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-03T20:27:19.574 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-03T20:27:19.574 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.compaction_readahead_size: 0 2024-05-03T20:27:19.574 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.max_background_flushes: -1 2024-05-03T20:27:19.574 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Compression algorithms supported: 2024-05-03T20:27:19.574 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-03T20:27:19.574 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: kZSTD supported: 0 2024-05-03T20:27:19.574 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: kXpressCompression supported: 0 2024-05-03T20:27:19.575 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: kLZ4HCCompression supported: 1 2024-05-03T20:27:19.575 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: kZlibCompression supported: 1 2024-05-03T20:27:19.575 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: kSnappyCompression supported: 1 2024-05-03T20:27:19.575 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: kLZ4Compression supported: 1 2024-05-03T20:27:19.575 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: kBZip2Compression supported: 0 2024-05-03T20:27:19.575 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-03T20:27:19.575 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-03T20:27:19.575 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-03T20:27:19.575 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-03T20:27:19.575 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-03T20:27:19.575 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.merge_operator: 2024-05-03T20:27:19.575 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.compaction_filter: None 2024-05-03T20:27:19.575 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.compaction_filter_factory: None 2024-05-03T20:27:19.575 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.sst_partitioner_factory: None 2024-05-03T20:27:19.575 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-03T20:27:19.575 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-03T20:27:19.576 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556522fb9f60) 2024-05-03T20:27:19.576 INFO:journalctl@ceph.mon.c.smithi204.stdout: cache_index_and_filter_blocks: 1 2024-05-03T20:27:19.576 INFO:journalctl@ceph.mon.c.smithi204.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-03T20:27:19.576 INFO:journalctl@ceph.mon.c.smithi204.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-03T20:27:19.576 INFO:journalctl@ceph.mon.c.smithi204.stdout: pin_top_level_index_and_filter: 1 2024-05-03T20:27:19.576 INFO:journalctl@ceph.mon.c.smithi204.stdout: index_type: 0 2024-05-03T20:27:19.576 INFO:journalctl@ceph.mon.c.smithi204.stdout: data_block_index_type: 0 2024-05-03T20:27:19.576 INFO:journalctl@ceph.mon.c.smithi204.stdout: index_shortening: 1 2024-05-03T20:27:19.576 INFO:journalctl@ceph.mon.c.smithi204.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-03T20:27:19.576 INFO:journalctl@ceph.mon.c.smithi204.stdout: checksum: 4 2024-05-03T20:27:19.576 INFO:journalctl@ceph.mon.c.smithi204.stdout: no_block_cache: 0 2024-05-03T20:27:19.576 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache: 0x556522fbd1f0 2024-05-03T20:27:19.576 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache_name: BinnedLRUCache 2024-05-03T20:27:19.576 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache_options: 2024-05-03T20:27:19.576 INFO:journalctl@ceph.mon.c.smithi204.stdout: capacity : 536870912 2024-05-03T20:27:19.577 INFO:journalctl@ceph.mon.c.smithi204.stdout: num_shard_bits : 4 2024-05-03T20:27:19.577 INFO:journalctl@ceph.mon.c.smithi204.stdout: strict_capacity_limit : 0 2024-05-03T20:27:19.577 INFO:journalctl@ceph.mon.c.smithi204.stdout: high_pri_pool_ratio: 0.000 2024-05-03T20:27:19.577 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache_compressed: (nil) 2024-05-03T20:27:19.577 INFO:journalctl@ceph.mon.c.smithi204.stdout: persistent_cache: (nil) 2024-05-03T20:27:19.577 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_size: 4096 2024-05-03T20:27:19.577 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_size_deviation: 10 2024-05-03T20:27:19.577 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_restart_interval: 16 2024-05-03T20:27:19.577 INFO:journalctl@ceph.mon.c.smithi204.stdout: index_block_restart_interval: 1 2024-05-03T20:27:19.577 INFO:journalctl@ceph.mon.c.smithi204.stdout: metadata_block_size: 4096 2024-05-03T20:27:19.577 INFO:journalctl@ceph.mon.c.smithi204.stdout: partition_filters: 0 2024-05-03T20:27:19.577 INFO:journalctl@ceph.mon.c.smithi204.stdout: use_delta_encoding: 1 2024-05-03T20:27:19.577 INFO:journalctl@ceph.mon.c.smithi204.stdout: filter_policy: bloomfilter 2024-05-03T20:27:19.577 INFO:journalctl@ceph.mon.c.smithi204.stdout: whole_key_filtering: 1 2024-05-03T20:27:19.577 INFO:journalctl@ceph.mon.c.smithi204.stdout: verify_compression: 0 2024-05-03T20:27:19.577 INFO:journalctl@ceph.mon.c.smithi204.stdout: read_amp_bytes_per_bit: 0 2024-05-03T20:27:19.578 INFO:journalctl@ceph.mon.c.smithi204.stdout: format_version: 5 2024-05-03T20:27:19.578 INFO:journalctl@ceph.mon.c.smithi204.stdout: enable_index_compression: 1 2024-05-03T20:27:19.578 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_align: 0 2024-05-03T20:27:19.578 INFO:journalctl@ceph.mon.c.smithi204.stdout: max_auto_readahead_size: 262144 2024-05-03T20:27:19.578 INFO:journalctl@ceph.mon.c.smithi204.stdout: prepopulate_block_cache: 0 2024-05-03T20:27:19.578 INFO:journalctl@ceph.mon.c.smithi204.stdout: initial_auto_readahead_size: 8192 2024-05-03T20:27:19.578 INFO:journalctl@ceph.mon.c.smithi204.stdout: num_file_reads_for_auto_readahead: 2 2024-05-03T20:27:19.578 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.write_buffer_size: 33554432 2024-05-03T20:27:19.578 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.max_write_buffer_number: 2 2024-05-03T20:27:19.578 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.compression: NoCompression 2024-05-03T20:27:19.578 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.bottommost_compression: Disabled 2024-05-03T20:27:19.578 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.prefix_extractor: nullptr 2024-05-03T20:27:19.578 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-03T20:27:19.578 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.num_levels: 7 2024-05-03T20:27:19.578 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-03T20:27:19.579 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-03T20:27:19.579 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-03T20:27:19.579 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-03T20:27:19.579 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-03T20:27:19.579 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-03T20:27:19.579 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-03T20:27:19.579 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-03T20:27:19.579 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-03T20:27:19.579 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-03T20:27:19.579 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-03T20:27:19.579 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-03T20:27:19.579 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-03T20:27:19.579 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.compression_opts.level: 32767 2024-05-03T20:27:19.579 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.compression_opts.strategy: 0 2024-05-03T20:27:19.579 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-03T20:27:19.580 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-03T20:27:19.580 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-03T20:27:19.580 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-03T20:27:19.580 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.compression_opts.enabled: false 2024-05-03T20:27:19.580 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-03T20:27:19.580 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-03T20:27:19.580 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-03T20:27:19.580 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-03T20:27:19.580 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.target_file_size_base: 67108864 2024-05-03T20:27:19.580 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-03T20:27:19.580 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-03T20:27:19.580 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-03T20:27:19.580 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-03T20:27:19.580 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-03T20:27:19.580 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-03T20:27:19.580 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-03T20:27:19.581 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-03T20:27:19.581 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-03T20:27:19.581 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-03T20:27:19.581 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-03T20:27:19.581 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-03T20:27:19.581 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-03T20:27:19.581 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-03T20:27:19.581 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.arena_block_size: 1048576 2024-05-03T20:27:19.581 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-03T20:27:19.581 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-03T20:27:19.581 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.disable_auto_compactions: 0 2024-05-03T20:27:19.581 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-03T20:27:19.581 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-03T20:27:19.581 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-03T20:27:19.582 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-03T20:27:19.582 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-03T20:27:19.582 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-03T20:27:19.582 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-03T20:27:19.582 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-03T20:27:19.582 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-03T20:27:19.582 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-03T20:27:19.582 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-03T20:27:19.582 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.inplace_update_support: 0 2024-05-03T20:27:19.582 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-03T20:27:19.582 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-03T20:27:19.582 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-03T20:27:19.582 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-03T20:27:19.582 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.bloom_locality: 0 2024-05-03T20:27:19.583 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.max_successive_merges: 0 2024-05-03T20:27:19.583 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-03T20:27:19.583 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.paranoid_file_checks: 0 2024-05-03T20:27:19.583 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.force_consistency_checks: 1 2024-05-03T20:27:19.583 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.report_bg_io_stats: 0 2024-05-03T20:27:19.583 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.ttl: 2592000 2024-05-03T20:27:19.583 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-03T20:27:19.583 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-03T20:27:19.583 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-03T20:27:19.583 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.enable_blob_files: false 2024-05-03T20:27:19.583 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.min_blob_size: 0 2024-05-03T20:27:19.583 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.blob_file_size: 268435456 2024-05-03T20:27:19.583 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-03T20:27:19.583 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-03T20:27:19.583 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-03T20:27:19.583 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-03T20:27:19.584 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-03T20:27:19.584 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.blob_file_starting_level: 0 2024-05-03T20:27:19.584 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-03T20:27:19.584 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-03T20:27:19.584 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-03T20:27:19.584 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d26b50b4-45d5-4168-b9e4-a2448c812b9c 2024-05-03T20:27:19.584 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714768039236144, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-03T20:27:19.584 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-03T20:27:19.584 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714768039236694, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714768039, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d26b50b4-45d5-4168-b9e4-a2448c812b9c", "db_session_id": "12A0K61K6GW34ITONA1F", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-03T20:27:19.584 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714768039236797, "job": 1, "event": "recovery_finished"} 2024-05-03T20:27:19.584 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-03T20:27:19.584 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-03T20:27:19.584 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5565230da000 2024-05-03T20:27:19.584 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: DB pointer 0x5565230c6000 2024-05-03T20:27:19.584 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-03T20:27:19.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-03T20:27:19.585 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** DB Stats ** 2024-05-03T20:27:19.585 INFO:journalctl@ceph.mon.c.smithi204.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-03T20:27:19.585 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-03T20:27:19.585 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-03T20:27:19.585 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-03T20:27:19.585 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-03T20:27:19.585 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-03T20:27:19.585 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-03T20:27:19.585 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-05-03T20:27:19.585 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** Compaction Stats [default] ** 2024-05-03T20:27:19.585 INFO:journalctl@ceph.mon.c.smithi204.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-03T20:27:19.585 INFO:journalctl@ceph.mon.c.smithi204.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-03T20:27:19.585 INFO:journalctl@ceph.mon.c.smithi204.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T20:27:19.586 INFO:journalctl@ceph.mon.c.smithi204.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T20:27:19.586 INFO:journalctl@ceph.mon.c.smithi204.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T20:27:19.586 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-05-03T20:27:19.586 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** Compaction Stats [default] ** 2024-05-03T20:27:19.586 INFO:journalctl@ceph.mon.c.smithi204.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-03T20:27:19.586 INFO:journalctl@ceph.mon.c.smithi204.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-03T20:27:19.586 INFO:journalctl@ceph.mon.c.smithi204.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T20:27:19.586 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-05-03T20:27:19.586 INFO:journalctl@ceph.mon.c.smithi204.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-03T20:27:19.586 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-05-03T20:27:19.586 INFO:journalctl@ceph.mon.c.smithi204.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-03T20:27:19.586 INFO:journalctl@ceph.mon.c.smithi204.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-03T20:27:19.586 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-03T20:27:19.586 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-03T20:27:19.587 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-03T20:27:19.587 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-03T20:27:19.587 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative compaction: 0.00 GB write, 0.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-03T20:27:19.587 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval compaction: 0.00 GB write, 0.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-03T20:27:19.587 INFO:journalctl@ceph.mon.c.smithi204.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-03T20:27:19.587 INFO:journalctl@ceph.mon.c.smithi204.stdout: Block cache BinnedLRUCache@0x556522fbd1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-05-03T20:27:19.587 INFO:journalctl@ceph.mon.c.smithi204.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-03T20:27:19.587 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-05-03T20:27:19.587 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-03T20:27:19.587 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-03T20:27:19.587 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: using public_addr v2:172.21.15.204:0/0 -> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-03T20:27:19.587 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: starting mon.c rank -1 at public addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] at bind addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:27:19.588 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: mon.c@-1(???) e0 preinit fsid 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:27:19.588 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: mon.c@-1(synchronizing).mds e1 new map 2024-05-03T20:27:19.588 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-03T20:27:19.588 INFO:journalctl@ceph.mon.c.smithi204.stdout: e1 2024-05-03T20:27:19.588 INFO:journalctl@ceph.mon.c.smithi204.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-03T20:27:19.588 INFO:journalctl@ceph.mon.c.smithi204.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-03T20:27:19.588 INFO:journalctl@ceph.mon.c.smithi204.stdout: legacy client fscid: -1 2024-05-03T20:27:19.588 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-05-03T20:27:19.588 INFO:journalctl@ceph.mon.c.smithi204.stdout: No filesystems configured 2024-05-03T20:27:19.588 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-03T20:27:19.588 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-03T20:27:19.588 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-03T20:27:19.588 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-03T20:27:19.588 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-03T20:27:19.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-03T20:27:19.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-03T20:27:19.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T20:27:19.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T20:27:19.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T20:27:19.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T20:27:19.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:19.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:19.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:27:19.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:19.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:19.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:19.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:19.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:27:19.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:19.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:27:19.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: Updating smithi196:/etc/ceph/ceph.conf 2024-05-03T20:27:19.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: Updating smithi196:/var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T20:27:19.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: Deploying cephadm binary to smithi204 2024-05-03T20:27:19.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-05-03T20:27:19.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: Updating smithi196:/var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T20:27:19.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:19.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:19.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:19.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:19.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:19.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:27:19.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:19.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: Added host smithi204 2024-05-03T20:27:19.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:19.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:19.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:19.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T20:27:19.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:19.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:19.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/4181499110' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-03T20:27:19.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/4181499110' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-03T20:27:19.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T20:27:19.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:19.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:19.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:19.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:19.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:27:19.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:19.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:27:19.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: Updating smithi204:/etc/ceph/ceph.conf 2024-05-03T20:27:19.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:19.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:19.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:19.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:19.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: Updating smithi204:/var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T20:27:19.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-05-03T20:27:19.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: Updating smithi204:/var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T20:27:19.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:19.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi181:172.21.15.181=a;smithi196:172.21.15.196=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:27:19.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: Saving service mon spec with placement smithi181:172.21.15.181=a;smithi196:172.21.15.196=b;smithi204:172.21.15.204=c;count:3 2024-05-03T20:27:19.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:19.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:27:19.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:19.593 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:27:19.593 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:19.593 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T20:27:19.593 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:19.593 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: Deploying daemon mon.c on smithi204 2024-05-03T20:27:19.593 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:19.593 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:19.593 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: from='client.? 172.21.15.204:0/3501517177' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T20:27:19.593 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 ceph-mon[33595]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-03T20:27:19.593 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 bash[33563]: f777a02cfbdfa59207ff2857b228cbc86f34e9b2db2897c0c514e9f535d1ee45 2024-05-03T20:27:19.593 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:19 smithi204 systemd[1]: Started Ceph mon.c for 1cd058ee-098b-11ef-bc97-c7b262605968. 2024-05-03T20:27:20.817 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-03T20:27:20.818 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-03T20:27:21.067 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T20:27:24.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T20:27:24.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: mon.a calling monitor election 2024-05-03T20:27:24.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T20:27:24.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:24.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T20:27:24.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T20:27:24.615 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: mon.c calling monitor election 2024-05-03T20:27:24.615 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:24.615 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T20:27:24.615 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T20:27:24.615 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T20:27:24.615 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-03T20:27:24.615 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: monmap epoch 2 2024-05-03T20:27:24.615 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: fsid 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:27:24.615 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: last_changed 2024-05-03T20:27:19.262664+0000 2024-05-03T20:27:24.615 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: created 2024-05-03T20:25:36.511893+0000 2024-05-03T20:27:24.615 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: min_mon_release 19 (squid) 2024-05-03T20:27:24.615 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: election_strategy: 1 2024-05-03T20:27:24.616 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: 0: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.a 2024-05-03T20:27:24.616 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-05-03T20:27:24.616 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: fsmap 2024-05-03T20:27:24.616 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T20:27:24.616 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: mgrmap e13: x(active, since 50s) 2024-05-03T20:27:24.616 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: overall HEALTH_OK 2024-05-03T20:27:24.616 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:24.616 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:24.616 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:24.616 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T20:27:24.616 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:24 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:24.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T20:27:24.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: mon.a calling monitor election 2024-05-03T20:27:24.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T20:27:24.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:24.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T20:27:24.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T20:27:24.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: mon.c calling monitor election 2024-05-03T20:27:24.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:24.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T20:27:24.715 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T20:27:24.715 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T20:27:24.715 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-03T20:27:24.715 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: monmap epoch 2 2024-05-03T20:27:24.715 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: fsid 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:27:24.715 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: last_changed 2024-05-03T20:27:19.262664+0000 2024-05-03T20:27:24.715 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: created 2024-05-03T20:25:36.511893+0000 2024-05-03T20:27:24.715 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: min_mon_release 19 (squid) 2024-05-03T20:27:24.715 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: election_strategy: 1 2024-05-03T20:27:24.715 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: 0: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.a 2024-05-03T20:27:24.715 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-05-03T20:27:24.715 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: fsmap 2024-05-03T20:27:24.715 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T20:27:24.715 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: mgrmap e13: x(active, since 50s) 2024-05-03T20:27:24.715 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: overall HEALTH_OK 2024-05-03T20:27:24.716 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:24.716 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:24.716 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:24.716 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T20:27:24.716 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:24 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:25.583 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.583+0000 7f969d59b700 1 -- 172.21.15.204:0/962348020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96800035c0 msgr2=0x7f9680005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:25.583 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.583+0000 7f969d59b700 1 --2- 172.21.15.204:0/962348020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96800035c0 0x7f9680005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9698102f00 tx=0x7f968c00feb0 comp rx=0 tx=0).stop 2024-05-03T20:27:25.583 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.583+0000 7f969d59b700 1 -- 172.21.15.204:0/962348020 shutdown_connections 2024-05-03T20:27:25.583 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.583+0000 7f969d59b700 1 --2- 172.21.15.204:0/962348020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96800035c0 0x7f9680005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:25.583 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.583+0000 7f969d59b700 1 -- 172.21.15.204:0/962348020 >> 172.21.15.204:0/962348020 conn(0x7f96980feac0 msgr2=0x7f9698100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:27:25.584 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.583+0000 7f969d59b700 1 -- 172.21.15.204:0/962348020 shutdown_connections 2024-05-03T20:27:25.584 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.583+0000 7f969d59b700 1 -- 172.21.15.204:0/962348020 wait complete. 2024-05-03T20:27:25.584 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:25 smithi204 ceph-mon[33595]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:25.584 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:25 smithi204 ceph-mon[33595]: Deploying daemon mon.b on smithi196 2024-05-03T20:27:25.584 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T20:27:25.586 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.584+0000 7f969d59b700 1 Processor -- start 2024-05-03T20:27:25.586 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.584+0000 7f969d59b700 1 -- start start 2024-05-03T20:27:25.586 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.584+0000 7f969d59b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96800035c0 0x7f969819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:25.586 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.584+0000 7f969d59b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f969819ebc0 0x7f9698198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:25.586 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.584+0000 7f969d59b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f969810af20 con 0x7f969819ebc0 2024-05-03T20:27:25.586 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.584+0000 7f969d59b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f969810ada0 con 0x7f96800035c0 2024-05-03T20:27:25.587 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.585+0000 7f9696ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96800035c0 0x7f969819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:25.587 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.585+0000 7f9696ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96800035c0 0x7f969819e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33028/0 (socket says 172.21.15.204:33028) 2024-05-03T20:27:25.587 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.585+0000 7f9696ffd700 1 -- 172.21.15.204:0/964298142 learned_addr learned my addr 172.21.15.204:0/964298142 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-03T20:27:25.587 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.585+0000 7f9696ffd700 1 -- 172.21.15.204:0/964298142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96800035c0 msgr2=0x7f969819e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-03T20:27:25.587 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.585+0000 7f9696ffd700 1 -- 172.21.15.204:0/964298142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96800035c0 msgr2=0x7f969819e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-03T20:27:25.587 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.585+0000 7f9696ffd700 1 --2- 172.21.15.204:0/964298142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96800035c0 0x7f969819e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-03T20:27:25.587 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.585+0000 7f96967fc700 1 --2- 172.21.15.204:0/964298142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f969819ebc0 0x7f9698198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:25.587 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.585+0000 7f9696ffd700 1 --2- 172.21.15.204:0/964298142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96800035c0 0x7f969819e680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T20:27:25.587 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.585+0000 7f96967fc700 1 -- 172.21.15.204:0/964298142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96800035c0 msgr2=0x7f969819e680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T20:27:25.587 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.585+0000 7f96967fc700 1 --2- 172.21.15.204:0/964298142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96800035c0 0x7f969819e680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:25.587 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.585+0000 7f96967fc700 1 -- 172.21.15.204:0/964298142 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f969819f200 con 0x7f969819ebc0 2024-05-03T20:27:25.587 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.585+0000 7f96967fc700 1 --2- 172.21.15.204:0/964298142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f969819ebc0 0x7f9698198850 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f968800b4d0 tx=0x7f968800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:27:25.587 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.585+0000 7f967ffff700 1 -- 172.21.15.204:0/964298142 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f9688011630 con 0x7f969819ebc0 2024-05-03T20:27:25.587 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.585+0000 7f969d59b700 1 -- 172.21.15.204:0/964298142 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9698198df0 con 0x7f969819ebc0 2024-05-03T20:27:25.588 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.585+0000 7f967ffff700 1 -- 172.21.15.204:0/964298142 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9688011c80 con 0x7f969819ebc0 2024-05-03T20:27:25.588 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.585+0000 7f967ffff700 1 -- 172.21.15.204:0/964298142 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f9688010da0 con 0x7f969819ebc0 2024-05-03T20:27:25.588 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.585+0000 7f969d59b700 1 -- 172.21.15.204:0/964298142 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9698199350 con 0x7f969819ebc0 2024-05-03T20:27:25.588 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.586+0000 7f967ffff700 1 -- 172.21.15.204:0/964298142 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f9688018020 con 0x7f969819ebc0 2024-05-03T20:27:25.588 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.586+0000 7f969d59b700 1 -- 172.21.15.204:0/964298142 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9698104710 con 0x7f969819ebc0 2024-05-03T20:27:25.588 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.587+0000 7f967ffff700 1 --2- 172.21.15.204:0/964298142 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f968403e2b0 0x7f9684040770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:25.588 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.587+0000 7f967ffff700 1 -- 172.21.15.204:0/964298142 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f96880523e0 con 0x7f969819ebc0 2024-05-03T20:27:25.588 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.587+0000 7f9696ffd700 1 --2- 172.21.15.204:0/964298142 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f968403e2b0 0x7f9684040770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:25.588 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.587+0000 7f9696ffd700 1 --2- 172.21.15.204:0/964298142 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f968403e2b0 0x7f9684040770 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f968c00fa60 tx=0x7f968c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:27:25.591 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.592+0000 7f967ffff700 1 -- 172.21.15.204:0/964298142 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f968801f020 con 0x7f969819ebc0 2024-05-03T20:27:25.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:25 smithi181 ceph-mon[29801]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:25.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:25 smithi181 ceph-mon[29801]: Deploying daemon mon.b on smithi196 2024-05-03T20:27:25.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T20:27:25.880 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.881+0000 7f969d59b700 1 -- 172.21.15.204:0/964298142 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f96981996c0 con 0x7f969819ebc0 2024-05-03T20:27:25.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.882+0000 7f967ffff700 1 -- 172.21.15.204:0/964298142 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f96880187b0 con 0x7f969819ebc0 2024-05-03T20:27:25.881 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-03T20:27:25.881 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":2,"fsid":"1cd058ee-098b-11ef-bc97-c7b262605968","modified":"2024-05-03T20:27:19.262664Z","created":"2024-05-03T20:25:36.511893Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:3300","nonce":0},{"type":"v1","addr":"172.21.15.181:6789","nonce":0}]},"addr":"172.21.15.181:6789/0","public_addr":"172.21.15.181:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-03T20:27:25.884 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.884+0000 7f969d59b700 1 -- 172.21.15.204:0/964298142 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f968403e2b0 msgr2=0x7f9684040770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:25.884 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.884+0000 7f969d59b700 1 --2- 172.21.15.204:0/964298142 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f968403e2b0 0x7f9684040770 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f968c00fa60 tx=0x7f968c005960 comp rx=0 tx=0).stop 2024-05-03T20:27:25.884 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.884+0000 7f969d59b700 1 -- 172.21.15.204:0/964298142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f969819ebc0 msgr2=0x7f9698198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:25.884 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.884+0000 7f969d59b700 1 --2- 172.21.15.204:0/964298142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f969819ebc0 0x7f9698198850 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f968800b4d0 tx=0x7f968800b890 comp rx=0 tx=0).stop 2024-05-03T20:27:25.884 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.884+0000 7f969d59b700 1 -- 172.21.15.204:0/964298142 shutdown_connections 2024-05-03T20:27:25.884 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.884+0000 7f969d59b700 1 --2- 172.21.15.204:0/964298142 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f968403e2b0 0x7f9684040770 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:25.884 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.884+0000 7f969d59b700 1 --2- 172.21.15.204:0/964298142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96800035c0 0x7f969819e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:25.885 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.884+0000 7f969d59b700 1 --2- 172.21.15.204:0/964298142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f969819ebc0 0x7f9698198850 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:25.885 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.884+0000 7f969d59b700 1 -- 172.21.15.204:0/964298142 >> 172.21.15.204:0/964298142 conn(0x7f96980feac0 msgr2=0x7f96980ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:27:25.885 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.884+0000 7f969d59b700 1 -- 172.21.15.204:0/964298142 shutdown_connections 2024-05-03T20:27:25.885 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:25.884+0000 7f969d59b700 1 -- 172.21.15.204:0/964298142 wait complete. 2024-05-03T20:27:25.885 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 2 2024-05-03T20:27:26.453 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:26 smithi204 ceph-mon[33595]: from='client.? 172.21.15.204:0/964298142' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T20:27:26.614 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:26 smithi181 ceph-mon[29801]: from='client.? 172.21.15.204:0/964298142' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T20:27:27.482 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-03T20:27:27.482 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-03T20:27:27.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:27 smithi204 ceph-mon[33595]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:27.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:27 smithi181 ceph-mon[29801]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:27.728 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T20:27:28.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:28 smithi196 systemd[1]: Started Ceph mon.b for 1cd058ee-098b-11ef-bc97-c7b262605968. 2024-05-03T20:27:33.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:33.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T20:27:33.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: mon.a calling monitor election 2024-05-03T20:27:33.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:27:33.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T20:27:33.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: mon.c calling monitor election 2024-05-03T20:27:33.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:27:33.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:33.465 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:27:33.465 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: mon.b calling monitor election 2024-05-03T20:27:33.465 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:27:33.465 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:33.465 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:27:33.465 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:27:33.465 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-03T20:27:33.465 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: monmap epoch 3 2024-05-03T20:27:33.465 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: fsid 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:27:33.465 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: last_changed 2024-05-03T20:27:28.092767+0000 2024-05-03T20:27:33.465 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: created 2024-05-03T20:25:36.511893+0000 2024-05-03T20:27:33.465 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: min_mon_release 19 (squid) 2024-05-03T20:27:33.465 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: election_strategy: 1 2024-05-03T20:27:33.465 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: 0: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.a 2024-05-03T20:27:33.466 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-05-03T20:27:33.466 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: 2: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.b 2024-05-03T20:27:33.466 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: fsmap 2024-05-03T20:27:33.466 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T20:27:33.466 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: mgrmap e13: x(active, since 59s) 2024-05-03T20:27:33.466 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: overall HEALTH_OK 2024-05-03T20:27:33.466 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:33.466 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:33.466 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:33.466 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:33 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:33.475 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.475+0000 7f2db4fe5700 1 -- 172.21.15.204:0/648696171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d9c0035c0 msgr2=0x7f2d9c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:33.475 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.475+0000 7f2db4fe5700 1 --2- 172.21.15.204:0/648696171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d9c0035c0 0x7f2d9c005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2db006b790 tx=0x7f2da000fe60 comp rx=0 tx=0).stop 2024-05-03T20:27:33.475 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.475+0000 7f2db4fe5700 1 -- 172.21.15.204:0/648696171 shutdown_connections 2024-05-03T20:27:33.475 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.475+0000 7f2db4fe5700 1 --2- 172.21.15.204:0/648696171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d9c0035c0 0x7f2d9c005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:33.475 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.475+0000 7f2db4fe5700 1 -- 172.21.15.204:0/648696171 >> 172.21.15.204:0/648696171 conn(0x7f2db00fd9a0 msgr2=0x7f2db00ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:27:33.475 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.475+0000 7f2db4fe5700 1 -- 172.21.15.204:0/648696171 shutdown_connections 2024-05-03T20:27:33.475 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.475+0000 7f2db4fe5700 1 -- 172.21.15.204:0/648696171 wait complete. 2024-05-03T20:27:33.476 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:33.476 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T20:27:33.476 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: mon.a calling monitor election 2024-05-03T20:27:33.476 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:27:33.476 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T20:27:33.476 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: mon.c calling monitor election 2024-05-03T20:27:33.476 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:27:33.476 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:33.477 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:27:33.477 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: mon.b calling monitor election 2024-05-03T20:27:33.477 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:27:33.477 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:33.477 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:27:33.477 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:27:33.477 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-03T20:27:33.477 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: monmap epoch 3 2024-05-03T20:27:33.477 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: fsid 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:27:33.477 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: last_changed 2024-05-03T20:27:28.092767+0000 2024-05-03T20:27:33.477 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: created 2024-05-03T20:25:36.511893+0000 2024-05-03T20:27:33.477 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: min_mon_release 19 (squid) 2024-05-03T20:27:33.477 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: election_strategy: 1 2024-05-03T20:27:33.477 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: 0: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.a 2024-05-03T20:27:33.478 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-05-03T20:27:33.478 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: 2: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.b 2024-05-03T20:27:33.478 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: fsmap 2024-05-03T20:27:33.478 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T20:27:33.478 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: mgrmap e13: x(active, since 59s) 2024-05-03T20:27:33.478 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: overall HEALTH_OK 2024-05-03T20:27:33.478 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:33.478 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:33.478 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:33.478 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:33 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:33.480 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.476+0000 7f2db4fe5700 1 Processor -- start 2024-05-03T20:27:33.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.476+0000 7f2db4fe5700 1 -- start start 2024-05-03T20:27:33.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.476+0000 7f2db4fe5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d9c0035c0 0x7f2db019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:33.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.477+0000 7f2db4fe5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2db019ed80 0x7f2db0198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:33.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.477+0000 7f2db4fe5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2db0198f40 0x7f2db01993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:33.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.477+0000 7f2db4fe5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2db010aeb0 con 0x7f2db019ed80 2024-05-03T20:27:33.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.477+0000 7f2db4fe5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2db010ad30 con 0x7f2d9c0035c0 2024-05-03T20:27:33.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.477+0000 7f2db4fe5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2db010b030 con 0x7f2db0198f40 2024-05-03T20:27:33.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.477+0000 7f2daed9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2db0198f40 0x7f2db01993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:33.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.477+0000 7f2daed9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2db0198f40 0x7f2db01993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33030/0 (socket says 172.21.15.204:33030) 2024-05-03T20:27:33.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.477+0000 7f2daed9d700 1 -- 172.21.15.204:0/4259492645 learned_addr learned my addr 172.21.15.204:0/4259492645 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-03T20:27:33.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.477+0000 7f2da5d9b700 1 --2- 172.21.15.204:0/4259492645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2db019ed80 0x7f2db0198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:33.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.477+0000 7f2dae59c700 1 --2- 172.21.15.204:0/4259492645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d9c0035c0 0x7f2db019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:33.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.477+0000 7f2daed9d700 1 -- 172.21.15.204:0/4259492645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2db0198f40 msgr2=0x7f2db01993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-05-03T20:27:33.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.477+0000 7f2daed9d700 1 -- 172.21.15.204:0/4259492645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2db0198f40 msgr2=0x7f2db01993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-03T20:27:33.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.477+0000 7f2daed9d700 1 --2- 172.21.15.204:0/4259492645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2db0198f40 0x7f2db01993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-03T20:27:33.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.477+0000 7f2daed9d700 1 --2- 172.21.15.204:0/4259492645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2db0198f40 0x7f2db01993f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T20:27:33.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.477+0000 7f2dae59c700 1 -- 172.21.15.204:0/4259492645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d9c0035c0 msgr2=0x7f2db019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-03T20:27:33.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.477+0000 7f2dae59c700 1 -- 172.21.15.204:0/4259492645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d9c0035c0 msgr2=0x7f2db019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-03T20:27:33.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.477+0000 7f2dae59c700 1 --2- 172.21.15.204:0/4259492645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d9c0035c0 0x7f2db019e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-03T20:27:33.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.477+0000 7f2dae59c700 1 --2- 172.21.15.204:0/4259492645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d9c0035c0 0x7f2db019e840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T20:27:33.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.477+0000 7f2da5d9b700 1 -- 172.21.15.204:0/4259492645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2db0198f40 msgr2=0x7f2db01993f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T20:27:33.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.477+0000 7f2da5d9b700 1 --2- 172.21.15.204:0/4259492645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2db0198f40 0x7f2db01993f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:33.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.477+0000 7f2da5d9b700 1 -- 172.21.15.204:0/4259492645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d9c0035c0 msgr2=0x7f2db019e840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T20:27:33.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.477+0000 7f2da5d9b700 1 --2- 172.21.15.204:0/4259492645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d9c0035c0 0x7f2db019e840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:33.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.477+0000 7f2da5d9b700 1 -- 172.21.15.204:0/4259492645 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db01a5710 con 0x7f2db019ed80 2024-05-03T20:27:33.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.478+0000 7f2da5d9b700 1 --2- 172.21.15.204:0/4259492645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2db019ed80 0x7f2db0198a00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2d9800b550 tx=0x7f2d9800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:27:33.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.478+0000 7f2da7fff700 1 -- 172.21.15.204:0/4259492645 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98004230 con 0x7f2db019ed80 2024-05-03T20:27:33.483 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.478+0000 7f2da7fff700 1 -- 172.21.15.204:0/4259492645 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d980043d0 con 0x7f2db019ed80 2024-05-03T20:27:33.483 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.478+0000 7f2db4fe5700 1 -- 172.21.15.204:0/4259492645 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db01a5940 con 0x7f2db019ed80 2024-05-03T20:27:33.483 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.478+0000 7f2da7fff700 1 -- 172.21.15.204:0/4259492645 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98011b90 con 0x7f2db019ed80 2024-05-03T20:27:33.483 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.478+0000 7f2db4fe5700 1 -- 172.21.15.204:0/4259492645 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db01a5ed0 con 0x7f2db019ed80 2024-05-03T20:27:33.483 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.479+0000 7f2da7fff700 1 -- 172.21.15.204:0/4259492645 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f2d9801a400 con 0x7f2db019ed80 2024-05-03T20:27:33.483 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.479+0000 7f2da7fff700 1 --2- 172.21.15.204:0/4259492645 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f2d9003e9a0 0x7f2d90040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:33.483 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.479+0000 7f2da7fff700 1 -- 172.21.15.204:0/4259492645 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2d980522b0 con 0x7f2db019ed80 2024-05-03T20:27:33.483 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.479+0000 7f2db4fe5700 1 -- 172.21.15.204:0/4259492645 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2db0064b00 con 0x7f2db019ed80 2024-05-03T20:27:33.483 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.479+0000 7f2dae59c700 1 --2- 172.21.15.204:0/4259492645 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f2d9003e9a0 0x7f2d90040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:33.483 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.480+0000 7f2dae59c700 1 --2- 172.21.15.204:0/4259492645 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f2d9003e9a0 0x7f2d90040e60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2da000fa10 tx=0x7f2da0005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:27:33.484 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.484+0000 7f2da7fff700 1 -- 172.21.15.204:0/4259492645 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2d98020740 con 0x7f2db019ed80 2024-05-03T20:27:33.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:33.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T20:27:33.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: mon.a calling monitor election 2024-05-03T20:27:33.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:27:33.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T20:27:33.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: mon.c calling monitor election 2024-05-03T20:27:33.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:27:33.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:33.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:27:33.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: mon.b calling monitor election 2024-05-03T20:27:33.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:27:33.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:33.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:27:33.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:27:33.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-03T20:27:33.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: monmap epoch 3 2024-05-03T20:27:33.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: fsid 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:27:33.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: last_changed 2024-05-03T20:27:28.092767+0000 2024-05-03T20:27:33.609 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: created 2024-05-03T20:25:36.511893+0000 2024-05-03T20:27:33.609 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: min_mon_release 19 (squid) 2024-05-03T20:27:33.609 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: election_strategy: 1 2024-05-03T20:27:33.609 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: 0: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.a 2024-05-03T20:27:33.609 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-05-03T20:27:33.609 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: 2: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.b 2024-05-03T20:27:33.609 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: fsmap 2024-05-03T20:27:33.609 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T20:27:33.609 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: mgrmap e13: x(active, since 59s) 2024-05-03T20:27:33.609 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: overall HEALTH_OK 2024-05-03T20:27:33.609 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:33.609 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:33.610 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:33.610 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:33 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:33.792 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.792+0000 7f2db4fe5700 1 -- 172.21.15.204:0/4259492645 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f2db01a6160 con 0x7f2db019ed80 2024-05-03T20:27:33.792 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.793+0000 7f2da7fff700 1 -- 172.21.15.204:0/4259492645 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1289 (secure 0 0 0) 0x7f2d980151c0 con 0x7f2db019ed80 2024-05-03T20:27:33.793 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-03T20:27:33.793 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":3,"fsid":"1cd058ee-098b-11ef-bc97-c7b262605968","modified":"2024-05-03T20:27:28.092767Z","created":"2024-05-03T20:25:36.511893Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:3300","nonce":0},{"type":"v1","addr":"172.21.15.181:6789","nonce":0}]},"addr":"172.21.15.181:6789/0","public_addr":"172.21.15.181:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-03T20:27:33.796 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.796+0000 7f2db4fe5700 1 -- 172.21.15.204:0/4259492645 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f2d9003e9a0 msgr2=0x7f2d90040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:33.796 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.796+0000 7f2db4fe5700 1 --2- 172.21.15.204:0/4259492645 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f2d9003e9a0 0x7f2d90040e60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2da000fa10 tx=0x7f2da0005a90 comp rx=0 tx=0).stop 2024-05-03T20:27:33.796 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.796+0000 7f2db4fe5700 1 -- 172.21.15.204:0/4259492645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2db019ed80 msgr2=0x7f2db0198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:33.796 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.796+0000 7f2db4fe5700 1 --2- 172.21.15.204:0/4259492645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2db019ed80 0x7f2db0198a00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2d9800b550 tx=0x7f2d9800b910 comp rx=0 tx=0).stop 2024-05-03T20:27:33.796 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.796+0000 7f2db4fe5700 1 -- 172.21.15.204:0/4259492645 shutdown_connections 2024-05-03T20:27:33.796 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.796+0000 7f2db4fe5700 1 --2- 172.21.15.204:0/4259492645 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f2d9003e9a0 0x7f2d90040e60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:33.796 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.796+0000 7f2db4fe5700 1 --2- 172.21.15.204:0/4259492645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2db0198f40 0x7f2db01993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:33.796 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.796+0000 7f2db4fe5700 1 --2- 172.21.15.204:0/4259492645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d9c0035c0 0x7f2db019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:33.796 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.796+0000 7f2db4fe5700 1 --2- 172.21.15.204:0/4259492645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2db019ed80 0x7f2db0198a00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:33.796 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.796+0000 7f2db4fe5700 1 -- 172.21.15.204:0/4259492645 >> 172.21.15.204:0/4259492645 conn(0x7f2db00fd9a0 msgr2=0x7f2db0104f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:27:33.796 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.796+0000 7f2db4fe5700 1 -- 172.21.15.204:0/4259492645 shutdown_connections 2024-05-03T20:27:33.796 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:33.796+0000 7f2db4fe5700 1 -- 172.21.15.204:0/4259492645 wait complete. 2024-05-03T20:27:33.797 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 3 2024-05-03T20:27:34.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:34 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:27:34.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:34 smithi196 ceph-mon[32858]: from='client.? 172.21.15.204:0/4259492645' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T20:27:34.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:34 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:34.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:34 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:27:34.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:34 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:27:34.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:34 smithi204 ceph-mon[33595]: from='client.? 172.21.15.204:0/4259492645' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T20:27:34.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:34 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:34.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:34 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:27:34.393 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-03T20:27:34.393 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-03T20:27:34.420 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:34 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:27:34.420 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:34 smithi181 ceph-mon[29801]: from='client.? 172.21.15.204:0/4259492645' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T20:27:34.420 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:34 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:34.420 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:34 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:27:34.641 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:27:35.758 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:35 smithi181 ceph-mon[29801]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:35.758 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:35 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:35.758 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:35 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:35.758 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:35 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:35.758 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:35 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:35.758 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:35 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:35.758 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:35 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:27:35.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:35 smithi196 ceph-mon[32858]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:35.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:35 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:35.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:35 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:35.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:35 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:35.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:35 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:35.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:35 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:35.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:35 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:27:35.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:35 smithi204 ceph-mon[33595]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:35.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:35 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:35.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:35 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:35.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:35 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:35.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:35 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:35.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:35 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:35.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:35 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:27:36.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.221+0000 7fc4e66e8700 1 -- 172.21.15.181:0/1580956717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4e0102010 msgr2=0x7fc4e01023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:36.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.221+0000 7fc4e66e8700 1 --2- 172.21.15.181:0/1580956717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4e0102010 0x7fc4e01023f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009910 tx=0x7fc4d0009c20 comp rx=0 tx=0).stop 2024-05-03T20:27:36.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.221+0000 7fc4e66e8700 1 -- 172.21.15.181:0/1580956717 shutdown_connections 2024-05-03T20:27:36.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.221+0000 7fc4e66e8700 1 --2- 172.21.15.181:0/1580956717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4e0102010 0x7fc4e01023f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:36.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.221+0000 7fc4e66e8700 1 -- 172.21.15.181:0/1580956717 >> 172.21.15.181:0/1580956717 conn(0x7fc4e00fd8e0 msgr2=0x7fc4e00ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:27:36.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.221+0000 7fc4e66e8700 1 -- 172.21.15.181:0/1580956717 shutdown_connections 2024-05-03T20:27:36.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.221+0000 7fc4e66e8700 1 -- 172.21.15.181:0/1580956717 wait complete. 2024-05-03T20:27:36.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.222+0000 7fc4e66e8700 1 Processor -- start 2024-05-03T20:27:36.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.222+0000 7fc4e66e8700 1 -- start start 2024-05-03T20:27:36.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.222+0000 7fc4e66e8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4e0102010 0x7fc4e0103ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:36.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.222+0000 7fc4e66e8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4e0103ff0 0x7fc4e0104470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:36.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.222+0000 7fc4e66e8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4e01a4d50 0x7fc4e01a7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:36.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.222+0000 7fc4e66e8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e010c290 con 0x7fc4e01a4d50 2024-05-03T20:27:36.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.222+0000 7fc4e66e8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e010c110 con 0x7fc4e0102010 2024-05-03T20:27:36.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.222+0000 7fc4e66e8700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e010c410 con 0x7fc4e0103ff0 2024-05-03T20:27:36.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.222+0000 7fc4e4c85700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4e01a4d50 0x7fc4e01a7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:36.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.222+0000 7fc4e4c85700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4e01a4d50 0x7fc4e01a7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37028/0 (socket says 172.21.15.181:37028) 2024-05-03T20:27:36.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.222+0000 7fc4e4c85700 1 -- 172.21.15.181:0/3751998291 learned_addr learned my addr 172.21.15.181:0/3751998291 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:27:36.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.222+0000 7fc4df7fe700 1 --2- 172.21.15.181:0/3751998291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4e0103ff0 0x7fc4e0104470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:36.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.222+0000 7fc4dffff700 1 --2- 172.21.15.181:0/3751998291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4e0102010 0x7fc4e0103ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:36.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.223+0000 7fc4e4c85700 1 -- 172.21.15.181:0/3751998291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4e0103ff0 msgr2=0x7fc4e0104470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:36.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.223+0000 7fc4e4c85700 1 --2- 172.21.15.181:0/3751998291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4e0103ff0 0x7fc4e0104470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:36.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.223+0000 7fc4e4c85700 1 -- 172.21.15.181:0/3751998291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4e0102010 msgr2=0x7fc4e0103ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:36.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.223+0000 7fc4e4c85700 1 --2- 172.21.15.181:0/3751998291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4e0102010 0x7fc4e0103ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:36.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.223+0000 7fc4e4c85700 1 -- 172.21.15.181:0/3751998291 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e01a7870 con 0x7fc4e01a4d50 2024-05-03T20:27:36.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.223+0000 7fc4dffff700 1 --2- 172.21.15.181:0/3751998291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4e0102010 0x7fc4e0103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:27:36.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.223+0000 7fc4df7fe700 1 --2- 172.21.15.181:0/3751998291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4e0103ff0 0x7fc4e0104470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:27:36.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.223+0000 7fc4e4c85700 1 --2- 172.21.15.181:0/3751998291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4e01a4d50 0x7fc4e01a7140 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800ba50 tx=0x7fc4d800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:27:36.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.223+0000 7fc4dd7fa700 1 -- 172.21.15.181:0/3751998291 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8019070 con 0x7fc4e01a4d50 2024-05-03T20:27:36.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.223+0000 7fc4dd7fa700 1 -- 172.21.15.181:0/3751998291 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4d8014440 con 0x7fc4e01a4d50 2024-05-03T20:27:36.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.223+0000 7fc4e66e8700 1 -- 172.21.15.181:0/3751998291 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e01a7b60 con 0x7fc4e01a4d50 2024-05-03T20:27:36.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.223+0000 7fc4dd7fa700 1 -- 172.21.15.181:0/3751998291 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8007590 con 0x7fc4e01a4d50 2024-05-03T20:27:36.227 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.223+0000 7fc4e66e8700 1 -- 172.21.15.181:0/3751998291 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e01a8050 con 0x7fc4e01a4d50 2024-05-03T20:27:36.227 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.224+0000 7fc4e66e8700 1 -- 172.21.15.181:0/3751998291 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e0064b00 con 0x7fc4e01a4d50 2024-05-03T20:27:36.227 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.224+0000 7fc4dd7fa700 1 -- 172.21.15.181:0/3751998291 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fc4d801c070 con 0x7fc4e01a4d50 2024-05-03T20:27:36.227 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.224+0000 7fc4dd7fa700 1 --2- 172.21.15.181:0/3751998291 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fc4c803e9a0 0x7fc4c8040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:36.227 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.225+0000 7fc4dd7fa700 1 -- 172.21.15.181:0/3751998291 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc4d8051ec0 con 0x7fc4e01a4d50 2024-05-03T20:27:36.227 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.225+0000 7fc4dffff700 1 --2- 172.21.15.181:0/3751998291 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fc4c803e9a0 0x7fc4c8040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:36.227 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.225+0000 7fc4dffff700 1 --2- 172.21.15.181:0/3751998291 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fc4c803e9a0 0x7fc4c8040e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0006010 tx=0x7fc4d000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:27:36.229 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.230+0000 7fc4dd7fa700 1 -- 172.21.15.181:0/3751998291 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc4d8022170 con 0x7fc4e01a4d50 2024-05-03T20:27:36.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.449+0000 7fc4e66e8700 1 -- 172.21.15.181:0/3751998291 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fc4e01a8380 con 0x7fc4e01a4d50 2024-05-03T20:27:36.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.449+0000 7fc4dd7fa700 1 -- 172.21.15.181:0/3751998291 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7fc4d8012c40 con 0x7fc4e01a4d50 2024-05-03T20:27:36.450 INFO:teuthology.orchestra.run.smithi181.stdout:# minimal ceph.conf for 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:27:36.450 INFO:teuthology.orchestra.run.smithi181.stdout:[global] 2024-05-03T20:27:36.450 INFO:teuthology.orchestra.run.smithi181.stdout: fsid = 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:27:36.450 INFO:teuthology.orchestra.run.smithi181.stdout: mon_host = [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-05-03T20:27:36.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.452+0000 7fc4e66e8700 1 -- 172.21.15.181:0/3751998291 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fc4c803e9a0 msgr2=0x7fc4c8040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:36.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.452+0000 7fc4e66e8700 1 --2- 172.21.15.181:0/3751998291 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fc4c803e9a0 0x7fc4c8040e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0006010 tx=0x7fc4d000b540 comp rx=0 tx=0).stop 2024-05-03T20:27:36.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.452+0000 7fc4e66e8700 1 -- 172.21.15.181:0/3751998291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4e01a4d50 msgr2=0x7fc4e01a7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:36.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.452+0000 7fc4e66e8700 1 --2- 172.21.15.181:0/3751998291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4e01a4d50 0x7fc4e01a7140 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800ba50 tx=0x7fc4d800be10 comp rx=0 tx=0).stop 2024-05-03T20:27:36.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.452+0000 7fc4e66e8700 1 -- 172.21.15.181:0/3751998291 shutdown_connections 2024-05-03T20:27:36.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.452+0000 7fc4e66e8700 1 --2- 172.21.15.181:0/3751998291 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fc4c803e9a0 0x7fc4c8040e60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:36.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.452+0000 7fc4e66e8700 1 --2- 172.21.15.181:0/3751998291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4e01a4d50 0x7fc4e01a7140 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:36.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.452+0000 7fc4e66e8700 1 --2- 172.21.15.181:0/3751998291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4e0102010 0x7fc4e0103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:36.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.452+0000 7fc4e66e8700 1 --2- 172.21.15.181:0/3751998291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4e0103ff0 0x7fc4e0104470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:36.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.452+0000 7fc4e66e8700 1 -- 172.21.15.181:0/3751998291 >> 172.21.15.181:0/3751998291 conn(0x7fc4e00fd8e0 msgr2=0x7fc4e006b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:27:36.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.452+0000 7fc4e66e8700 1 -- 172.21.15.181:0/3751998291 shutdown_connections 2024-05-03T20:27:36.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:36.452+0000 7fc4e66e8700 1 -- 172.21.15.181:0/3751998291 wait complete. 2024-05-03T20:27:36.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:36 smithi181 ceph-mon[29801]: Updating smithi181:/etc/ceph/ceph.conf 2024-05-03T20:27:36.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:36 smithi181 ceph-mon[29801]: Updating smithi196:/etc/ceph/ceph.conf 2024-05-03T20:27:36.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:36 smithi181 ceph-mon[29801]: Updating smithi204:/etc/ceph/ceph.conf 2024-05-03T20:27:36.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:36 smithi181 ceph-mon[29801]: Updating smithi204:/var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T20:27:36.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:36 smithi181 ceph-mon[29801]: Updating smithi196:/var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T20:27:36.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:36 smithi181 ceph-mon[29801]: Updating smithi181:/var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T20:27:36.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:36.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:36.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:36.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:36.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:36.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:36.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:36.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:36.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:36.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:36.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:36.965 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T20:27:36.965 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T20:27:36.965 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:36.965 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:36 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3751998291' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:37.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:36 smithi196 ceph-mon[32858]: Updating smithi181:/etc/ceph/ceph.conf 2024-05-03T20:27:37.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:36 smithi196 ceph-mon[32858]: Updating smithi196:/etc/ceph/ceph.conf 2024-05-03T20:27:37.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:36 smithi196 ceph-mon[32858]: Updating smithi204:/etc/ceph/ceph.conf 2024-05-03T20:27:37.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:36 smithi196 ceph-mon[32858]: Updating smithi204:/var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T20:27:37.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:36 smithi196 ceph-mon[32858]: Updating smithi196:/var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T20:27:37.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:36 smithi196 ceph-mon[32858]: Updating smithi181:/var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T20:27:37.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:37.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:37.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:37.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:37.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:37.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:37.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:37.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:37.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:37.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:37.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:37.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T20:27:37.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T20:27:37.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:37.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:36 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3751998291' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:37.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:36 smithi204 ceph-mon[33595]: Updating smithi181:/etc/ceph/ceph.conf 2024-05-03T20:27:37.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:36 smithi204 ceph-mon[33595]: Updating smithi196:/etc/ceph/ceph.conf 2024-05-03T20:27:37.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:36 smithi204 ceph-mon[33595]: Updating smithi204:/etc/ceph/ceph.conf 2024-05-03T20:27:37.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:36 smithi204 ceph-mon[33595]: Updating smithi204:/var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T20:27:37.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:36 smithi204 ceph-mon[33595]: Updating smithi196:/var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T20:27:37.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:36 smithi204 ceph-mon[33595]: Updating smithi181:/var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T20:27:37.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:37.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:37.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:37.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:37.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:37.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:37.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:37.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:37.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:37.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:37.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:37.115 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T20:27:37.115 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T20:27:37.115 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:37.115 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:36 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3751998291' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:37.757 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:37 smithi181 ceph-mon[29801]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:37.757 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:37 smithi181 ceph-mon[29801]: Reconfiguring mon.a (unknown last config time)... 2024-05-03T20:27:37.757 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:37 smithi181 ceph-mon[29801]: Reconfiguring daemon mon.a on smithi181 2024-05-03T20:27:37.758 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:37 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:37.758 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:37 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:37.758 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:37 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T20:27:37.758 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:37 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T20:27:37.758 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:37 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:37.959 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-03T20:27:37.960 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-03T20:27:37.960 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.conf 2024-05-03T20:27:38.026 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-03T20:27:38.026 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-03T20:27:38.089 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-03T20:27:38.089 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.conf 2024-05-03T20:27:38.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:37 smithi196 ceph-mon[32858]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:38.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:37 smithi196 ceph-mon[32858]: Reconfiguring mon.a (unknown last config time)... 2024-05-03T20:27:38.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:37 smithi196 ceph-mon[32858]: Reconfiguring daemon mon.a on smithi181 2024-05-03T20:27:38.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:37 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:38.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:37 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:38.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:37 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T20:27:38.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:37 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T20:27:38.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:37 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:38.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:37 smithi204 ceph-mon[33595]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:38.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:37 smithi204 ceph-mon[33595]: Reconfiguring mon.a (unknown last config time)... 2024-05-03T20:27:38.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:37 smithi204 ceph-mon[33595]: Reconfiguring daemon mon.a on smithi181 2024-05-03T20:27:38.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:37 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:38.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:37 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:38.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:37 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T20:27:38.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:37 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T20:27:38.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:37 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:38.115 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-03T20:27:38.115 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-03T20:27:38.179 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-03T20:27:38.179 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.conf 2024-05-03T20:27:38.211 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-03T20:27:38.211 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-03T20:27:38.280 INFO:tasks.cephadm:Adding mgr.x on smithi181 2024-05-03T20:27:38.280 INFO:tasks.cephadm:Adding mgr.y on smithi196 2024-05-03T20:27:38.280 INFO:tasks.cephadm:Adding mgr.z on smithi204 2024-05-03T20:27:38.280 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph orch apply mgr '3;smithi181=x;smithi196=y;smithi204=z' 2024-05-03T20:27:38.557 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T20:27:38.956 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:38 smithi204 ceph-mon[33595]: Reconfiguring mon.b (monmap changed)... 2024-05-03T20:27:38.956 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:38 smithi204 ceph-mon[33595]: Reconfiguring daemon mon.b on smithi196 2024-05-03T20:27:38.972 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:38 smithi196 ceph-mon[32858]: Reconfiguring mon.b (monmap changed)... 2024-05-03T20:27:38.972 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:38 smithi196 ceph-mon[32858]: Reconfiguring daemon mon.b on smithi196 2024-05-03T20:27:39.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:38 smithi181 ceph-mon[29801]: Reconfiguring mon.b (monmap changed)... 2024-05-03T20:27:39.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:38 smithi181 ceph-mon[29801]: Reconfiguring daemon mon.b on smithi196 2024-05-03T20:27:39.850 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:39 smithi204 ceph-mon[33595]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:39.850 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:39 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:39.850 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:39 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:39.850 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:39 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T20:27:39.850 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:39 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T20:27:39.850 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:39 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:40.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:39 smithi196 ceph-mon[32858]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:40.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:39 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:40.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:39 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:40.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:39 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T20:27:40.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:39 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T20:27:40.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:39 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:40.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:39 smithi181 ceph-mon[29801]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:40.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:39 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:40.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:39 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:40.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:39 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T20:27:40.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:39 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T20:27:40.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:39 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:40.224 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.225+0000 7f23da736700 1 -- 172.21.15.204:0/3634831458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23d4103030 msgr2=0x7f23d4103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:40.225 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.225+0000 7f23da736700 1 --2- 172.21.15.204:0/3634831458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23d4103030 0x7f23d4103450 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009910 tx=0x7f23c4009c20 comp rx=0 tx=0).stop 2024-05-03T20:27:40.225 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.225+0000 7f23da736700 1 -- 172.21.15.204:0/3634831458 shutdown_connections 2024-05-03T20:27:40.225 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.225+0000 7f23da736700 1 --2- 172.21.15.204:0/3634831458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23d4103030 0x7f23d4103450 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:40.225 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.225+0000 7f23da736700 1 -- 172.21.15.204:0/3634831458 >> 172.21.15.204:0/3634831458 conn(0x7f23d40fe5d0 msgr2=0x7f23d4100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:27:40.225 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.225+0000 7f23da736700 1 -- 172.21.15.204:0/3634831458 shutdown_connections 2024-05-03T20:27:40.225 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.225+0000 7f23da736700 1 -- 172.21.15.204:0/3634831458 wait complete. 2024-05-03T20:27:40.228 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.226+0000 7f23da736700 1 Processor -- start 2024-05-03T20:27:40.228 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.226+0000 7f23da736700 1 -- start start 2024-05-03T20:27:40.228 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.226+0000 7f23da736700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23d4103030 0x7f23d4198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:40.228 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.226+0000 7f23da736700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23d4198cb0 0x7f23d419dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:40.228 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.226+0000 7f23da736700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23d4199130 0x7f23d41a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:40.228 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.226+0000 7f23da736700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f23d4108f70 con 0x7f23d4199130 2024-05-03T20:27:40.228 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.226+0000 7f23da736700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f23d4108df0 con 0x7f23d4103030 2024-05-03T20:27:40.228 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.226+0000 7f23da736700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f23d41090f0 con 0x7f23d4198cb0 2024-05-03T20:27:40.228 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.226+0000 7f23d37fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23d4198cb0 0x7f23d419dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:40.228 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.226+0000 7f23d37fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23d4198cb0 0x7f23d419dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:55772/0 (socket says 172.21.15.204:55772) 2024-05-03T20:27:40.228 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.226+0000 7f23d37fe700 1 -- 172.21.15.204:0/2986460765 learned_addr learned my addr 172.21.15.204:0/2986460765 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-03T20:27:40.228 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.226+0000 7f23d8cd3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23d4199130 0x7f23d41a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:40.229 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.227+0000 7f23d3fff700 1 --2- 172.21.15.204:0/2986460765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23d4103030 0x7f23d4198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:40.229 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.227+0000 7f23d37fe700 1 -- 172.21.15.204:0/2986460765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23d4103030 msgr2=0x7f23d4198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:40.229 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.227+0000 7f23d37fe700 1 --2- 172.21.15.204:0/2986460765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23d4103030 0x7f23d4198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:40.229 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.227+0000 7f23d37fe700 1 -- 172.21.15.204:0/2986460765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23d4199130 msgr2=0x7f23d41a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:40.229 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.227+0000 7f23d37fe700 1 --2- 172.21.15.204:0/2986460765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23d4199130 0x7f23d41a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:40.229 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.227+0000 7f23d37fe700 1 -- 172.21.15.204:0/2986460765 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d41a0b80 con 0x7f23d4198cb0 2024-05-03T20:27:40.229 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.227+0000 7f23d8cd3700 1 --2- 172.21.15.204:0/2986460765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23d4199130 0x7f23d41a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T20:27:40.229 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.227+0000 7f23d37fe700 1 --2- 172.21.15.204:0/2986460765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23d4198cb0 0x7f23d419dd20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f23c800b780 tx=0x7f23c800ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:27:40.229 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.227+0000 7f23d17fa700 1 -- 172.21.15.204:0/2986460765 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8004230 con 0x7f23d4198cb0 2024-05-03T20:27:40.229 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.227+0000 7f23d17fa700 1 -- 172.21.15.204:0/2986460765 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23c80043d0 con 0x7f23d4198cb0 2024-05-03T20:27:40.229 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.227+0000 7f23da736700 1 -- 172.21.15.204:0/2986460765 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d41a0e70 con 0x7f23d4198cb0 2024-05-03T20:27:40.229 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.227+0000 7f23d17fa700 1 -- 172.21.15.204:0/2986460765 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8011ad0 con 0x7f23d4198cb0 2024-05-03T20:27:40.229 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.227+0000 7f23da736700 1 -- 172.21.15.204:0/2986460765 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d41a13a0 con 0x7f23d4198cb0 2024-05-03T20:27:40.229 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.228+0000 7f23d17fa700 1 -- 172.21.15.204:0/2986460765 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f23c8011cf0 con 0x7f23d4198cb0 2024-05-03T20:27:40.229 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.228+0000 7f23da736700 1 -- 172.21.15.204:0/2986460765 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23d4064b00 con 0x7f23d4198cb0 2024-05-03T20:27:40.229 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.228+0000 7f23d17fa700 1 --2- 172.21.15.204:0/2986460765 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f23bc03e9a0 0x7f23bc040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:40.230 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.228+0000 7f23d17fa700 1 -- 172.21.15.204:0/2986460765 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f23c80522d0 con 0x7f23d4198cb0 2024-05-03T20:27:40.230 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.228+0000 7f23d3fff700 1 --2- 172.21.15.204:0/2986460765 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f23bc03e9a0 0x7f23bc040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:40.230 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.229+0000 7f23d3fff700 1 --2- 172.21.15.204:0/2986460765 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f23bc03e9a0 0x7f23bc040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f23c40050f0 tx=0x7f23c401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:27:40.233 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.234+0000 7f23d17fa700 1 -- 172.21.15.204:0/2986460765 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f23c8020740 con 0x7f23d4198cb0 2024-05-03T20:27:40.455 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.456+0000 7f23da736700 1 -- 172.21.15.204:0/2986460765 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi181=x;smithi196=y;smithi204=z", "target": ["mon-mgr", ""]}) -- 0x7f23d40638e0 con 0x7f23bc03e9a0 2024-05-03T20:27:40.462 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.463+0000 7f23d17fa700 1 -- 172.21.15.204:0/2986460765 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f23d40638e0 con 0x7f23bc03e9a0 2024-05-03T20:27:40.463 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mgr update... 2024-05-03T20:27:40.465 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.465+0000 7f23da736700 1 -- 172.21.15.204:0/2986460765 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f23bc03e9a0 msgr2=0x7f23bc040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:40.465 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.465+0000 7f23da736700 1 --2- 172.21.15.204:0/2986460765 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f23bc03e9a0 0x7f23bc040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f23c40050f0 tx=0x7f23c401a040 comp rx=0 tx=0).stop 2024-05-03T20:27:40.465 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.465+0000 7f23da736700 1 -- 172.21.15.204:0/2986460765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23d4198cb0 msgr2=0x7f23d419dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:40.465 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.465+0000 7f23da736700 1 --2- 172.21.15.204:0/2986460765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23d4198cb0 0x7f23d419dd20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f23c800b780 tx=0x7f23c800ba90 comp rx=0 tx=0).stop 2024-05-03T20:27:40.465 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.465+0000 7f23da736700 1 -- 172.21.15.204:0/2986460765 shutdown_connections 2024-05-03T20:27:40.465 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.465+0000 7f23da736700 1 --2- 172.21.15.204:0/2986460765 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f23bc03e9a0 0x7f23bc040e60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:40.465 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.465+0000 7f23da736700 1 --2- 172.21.15.204:0/2986460765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23d4199130 0x7f23d41a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:40.465 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.465+0000 7f23da736700 1 --2- 172.21.15.204:0/2986460765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23d4103030 0x7f23d4198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:40.465 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.465+0000 7f23da736700 1 --2- 172.21.15.204:0/2986460765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23d4198cb0 0x7f23d419dd20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:40.466 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.465+0000 7f23da736700 1 -- 172.21.15.204:0/2986460765 >> 172.21.15.204:0/2986460765 conn(0x7f23d40fe5d0 msgr2=0x7f23d40ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:27:40.466 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.465+0000 7f23da736700 1 -- 172.21.15.204:0/2986460765 shutdown_connections 2024-05-03T20:27:40.466 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:27:40.465+0000 7f23da736700 1 -- 172.21.15.204:0/2986460765 wait complete. 2024-05-03T20:27:41.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:40 smithi196 ceph-mon[32858]: Reconfiguring mon.c (monmap changed)... 2024-05-03T20:27:41.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:40 smithi196 ceph-mon[32858]: Reconfiguring daemon mon.c on smithi204 2024-05-03T20:27:41.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:40 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:41.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:40 smithi204 ceph-mon[33595]: Reconfiguring mon.c (monmap changed)... 2024-05-03T20:27:41.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:40 smithi204 ceph-mon[33595]: Reconfiguring daemon mon.c on smithi204 2024-05-03T20:27:41.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:40 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:41.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:40 smithi181 ceph-mon[29801]: Reconfiguring mon.c (monmap changed)... 2024-05-03T20:27:41.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:40 smithi181 ceph-mon[29801]: Reconfiguring daemon mon.c on smithi204 2024-05-03T20:27:41.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:40 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:41.238 DEBUG:teuthology.orchestra.run.smithi196:mgr.y> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mgr.y.service 2024-05-03T20:27:41.241 DEBUG:teuthology.orchestra.run.smithi204:mgr.z> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mgr.z.service 2024-05-03T20:27:41.243 INFO:tasks.cephadm:Deploying OSDs... 2024-05-03T20:27:41.244 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-03T20:27:41.244 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/scratch_devs of=/dev/stdout 2024-05-03T20:27:41.263 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-03T20:27:41.263 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_1 2024-05-03T20:27:41.319 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-03T20:27:41.319 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T20:27:41.320 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-03T20:27:41.320 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T20:27:41.320 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T20:27:41.320 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-05-03 20:26:46.699894763 +0000 2024-05-03T20:27:41.320 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-05-03 20:21:35.623398078 +0000 2024-05-03T20:27:41.320 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-05-03 20:21:35.623398078 +0000 2024-05-03T20:27:41.320 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-05-03 20:21:35.623398078 +0000 2024-05-03T20:27:41.320 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-03T20:27:41.385 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-05-03T20:27:41.386 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-05-03T20:27:41.386 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000205811 s, 2.5 MB/s 2024-05-03T20:27:41.387 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-03T20:27:41.446 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_2 2024-05-03T20:27:41.502 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-03T20:27:41.502 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T20:27:41.502 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-05-03T20:27:41.502 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T20:27:41.503 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T20:27:41.503 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-05-03 20:26:46.733894096 +0000 2024-05-03T20:27:41.503 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-05-03 20:21:35.971390829 +0000 2024-05-03T20:27:41.503 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-05-03 20:21:35.971390829 +0000 2024-05-03T20:27:41.503 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-05-03 20:21:35.971390829 +0000 2024-05-03T20:27:41.503 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-03T20:27:41.568 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-05-03T20:27:41.568 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-05-03T20:27:41.568 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000215524 s, 2.4 MB/s 2024-05-03T20:27:41.569 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-03T20:27:41.627 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_3 2024-05-03T20:27:41.681 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-03T20:27:41.682 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T20:27:41.682 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-03T20:27:41.682 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T20:27:41.682 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T20:27:41.682 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-05-03 20:26:46.765893469 +0000 2024-05-03T20:27:41.682 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-05-03 20:21:36.312383726 +0000 2024-05-03T20:27:41.682 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-05-03 20:21:36.312383726 +0000 2024-05-03T20:27:41.682 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-05-03 20:21:36.312383726 +0000 2024-05-03T20:27:41.682 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-03T20:27:41.743 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-05-03T20:27:41.744 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-05-03T20:27:41.744 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000220877 s, 2.3 MB/s 2024-05-03T20:27:41.745 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-03T20:27:41.801 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_4 2024-05-03T20:27:41.855 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-03T20:27:41.855 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T20:27:41.855 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-03T20:27:41.856 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T20:27:41.856 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T20:27:41.856 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-05-03 20:26:46.797892841 +0000 2024-05-03T20:27:41.856 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-05-03 20:21:36.644376809 +0000 2024-05-03T20:27:41.856 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-05-03 20:21:36.644376809 +0000 2024-05-03T20:27:41.856 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-05-03 20:21:36.644376809 +0000 2024-05-03T20:27:41.856 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-03T20:27:41.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:41 smithi204 ceph-mon[33595]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:41.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:41 smithi204 ceph-mon[33595]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi181=x;smithi196=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:27:41.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:41 smithi204 ceph-mon[33595]: Saving service mgr spec with placement smithi181=x;smithi196=y;smithi204=z;count:3 2024-05-03T20:27:41.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:41 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:41.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:41 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:41.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:41 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:27:41.918 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-05-03T20:27:41.918 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-05-03T20:27:41.918 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000170017 s, 3.0 MB/s 2024-05-03T20:27:41.919 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-03T20:27:41.975 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-05-03T20:27:41.975 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-05-03T20:27:41.994 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-03T20:27:41.994 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_1 2024-05-03T20:27:42.051 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-03T20:27:42.052 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T20:27:42.052 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-03T20:27:42.052 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T20:27:42.052 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T20:27:42.052 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-05-03 20:27:27.583097815 +0000 2024-05-03T20:27:42.052 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-05-03 20:22:15.372057011 +0000 2024-05-03T20:27:42.052 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-05-03 20:22:15.372057011 +0000 2024-05-03T20:27:42.052 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-05-03 20:22:15.372057011 +0000 2024-05-03T20:27:42.052 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-03T20:27:42.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:41 smithi196 ceph-mon[32858]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:42.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:41 smithi196 ceph-mon[32858]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi181=x;smithi196=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:27:42.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:41 smithi196 ceph-mon[32858]: Saving service mgr spec with placement smithi181=x;smithi196=y;smithi204=z;count:3 2024-05-03T20:27:42.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:41 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:42.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:41 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:42.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:41 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:27:42.118 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-05-03T20:27:42.118 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-05-03T20:27:42.118 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000282234 s, 1.8 MB/s 2024-05-03T20:27:42.119 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-03T20:27:42.176 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_2 2024-05-03T20:27:42.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:41 smithi181 ceph-mon[29801]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:42.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:41 smithi181 ceph-mon[29801]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi181=x;smithi196=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:27:42.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:41 smithi181 ceph-mon[29801]: Saving service mgr spec with placement smithi181=x;smithi196=y;smithi204=z;count:3 2024-05-03T20:27:42.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:41 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:42.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:41 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:42.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:41 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:27:42.232 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-03T20:27:42.232 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T20:27:42.232 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-05-03T20:27:42.232 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T20:27:42.233 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T20:27:42.233 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-05-03 20:27:27.583097815 +0000 2024-05-03T20:27:42.233 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-05-03 20:22:15.721048765 +0000 2024-05-03T20:27:42.233 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-05-03 20:22:15.721048765 +0000 2024-05-03T20:27:42.233 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-05-03 20:22:15.721048765 +0000 2024-05-03T20:27:42.233 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-03T20:27:42.297 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-05-03T20:27:42.466 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-05-03T20:27:42.466 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000271855 s, 1.9 MB/s 2024-05-03T20:27:42.467 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-03T20:27:42.525 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_3 2024-05-03T20:27:42.583 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-03T20:27:42.583 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T20:27:42.583 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-03T20:27:42.583 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T20:27:42.583 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T20:27:42.583 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-05-03 20:27:27.584097793 +0000 2024-05-03T20:27:42.584 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-05-03 20:22:16.057040825 +0000 2024-05-03T20:27:42.584 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-05-03 20:22:16.057040825 +0000 2024-05-03T20:27:42.584 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-05-03 20:22:16.057040825 +0000 2024-05-03T20:27:42.584 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-03T20:27:42.647 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-05-03T20:27:42.648 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-05-03T20:27:42.648 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000247385 s, 2.1 MB/s 2024-05-03T20:27:42.649 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-03T20:27:42.706 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_4 2024-05-03T20:27:42.762 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-03T20:27:42.762 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T20:27:42.762 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-03T20:27:42.762 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T20:27:42.762 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T20:27:42.762 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-05-03 20:27:27.584097793 +0000 2024-05-03T20:27:42.763 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-05-03 20:22:16.405032602 +0000 2024-05-03T20:27:42.763 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-05-03 20:22:16.405032602 +0000 2024-05-03T20:27:42.763 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-05-03 20:22:16.405032602 +0000 2024-05-03T20:27:42.763 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-03T20:27:42.825 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-05-03T20:27:42.825 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-05-03T20:27:42.826 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000231796 s, 2.2 MB/s 2024-05-03T20:27:42.827 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-03T20:27:42.883 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-03T20:27:42.883 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/scratch_devs of=/dev/stdout 2024-05-03T20:27:42.899 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-03T20:27:42.899 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_1 2024-05-03T20:27:42.953 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-03T20:27:42.954 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T20:27:42.954 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-03T20:27:42.954 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T20:27:42.954 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T20:27:42.954 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-05-03 20:27:18.645478411 +0000 2024-05-03T20:27:42.954 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-05-03 20:21:53.290781912 +0000 2024-05-03T20:27:42.954 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-05-03 20:21:53.290781912 +0000 2024-05-03T20:27:42.954 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-05-03 20:21:53.290781912 +0000 2024-05-03T20:27:42.954 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-03T20:27:43.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:42 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:43.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:42 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:27:43.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:42 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:43.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:42 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T20:27:43.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:42 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T20:27:43.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:42 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T20:27:43.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:42 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:43.017 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-05-03T20:27:43.017 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-05-03T20:27:43.017 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000267699 s, 1.9 MB/s 2024-05-03T20:27:43.019 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-03T20:27:43.075 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_2 2024-05-03T20:27:43.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:42 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:43.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:42 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:27:43.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:42 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:43.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:42 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T20:27:43.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:42 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T20:27:43.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:42 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T20:27:43.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:42 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:43.131 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-03T20:27:43.131 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T20:27:43.131 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-03T20:27:43.131 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T20:27:43.131 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T20:27:43.131 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-05-03 20:27:18.645478411 +0000 2024-05-03T20:27:43.131 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-05-03 20:21:53.637777309 +0000 2024-05-03T20:27:43.131 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-05-03 20:21:53.637777309 +0000 2024-05-03T20:27:43.131 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-05-03 20:21:53.637777309 +0000 2024-05-03T20:27:43.131 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-03T20:27:43.195 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-05-03T20:27:43.195 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-05-03T20:27:43.196 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000219069 s, 2.3 MB/s 2024-05-03T20:27:43.197 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-03T20:27:43.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:42 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:43.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:42 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:27:43.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:42 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:43.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:42 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T20:27:43.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:42 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T20:27:43.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:42 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T20:27:43.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:42 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:43.252 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_3 2024-05-03T20:27:43.307 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-03T20:27:43.307 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T20:27:43.307 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-03T20:27:43.307 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T20:27:43.307 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T20:27:43.307 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-05-03 20:27:18.645478411 +0000 2024-05-03T20:27:43.307 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-05-03 20:21:53.979772772 +0000 2024-05-03T20:27:43.308 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-05-03 20:21:53.979772772 +0000 2024-05-03T20:27:43.308 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-05-03 20:21:53.979772772 +0000 2024-05-03T20:27:43.308 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-03T20:27:43.370 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-05-03T20:27:43.370 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-05-03T20:27:43.370 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000197759 s, 2.6 MB/s 2024-05-03T20:27:43.371 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-03T20:27:43.427 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_4 2024-05-03T20:27:43.481 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-03T20:27:43.481 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T20:27:43.481 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-03T20:27:43.482 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T20:27:43.482 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T20:27:43.482 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-05-03 20:27:18.646478398 +0000 2024-05-03T20:27:43.482 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-05-03 20:21:54.308768407 +0000 2024-05-03T20:27:43.482 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-05-03 20:21:54.308768407 +0000 2024-05-03T20:27:43.482 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-05-03 20:21:54.308768407 +0000 2024-05-03T20:27:43.482 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-03T20:27:43.545 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-05-03T20:27:43.545 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-05-03T20:27:43.545 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000192427 s, 2.7 MB/s 2024-05-03T20:27:43.546 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-03T20:27:43.601 INFO:tasks.cephadm:Deploying osd.0 on smithi181 with /dev/vg_nvme/lv_4... 2024-05-03T20:27:43.601 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-03T20:27:43.826 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:43 smithi204 ceph-mon[33595]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:43.826 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:43 smithi204 ceph-mon[33595]: Deploying daemon mgr.z on smithi204 2024-05-03T20:27:43.846 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:27:43.959 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:43 smithi181 ceph-mon[29801]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:43.960 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:43 smithi181 ceph-mon[29801]: Deploying daemon mgr.z on smithi204 2024-05-03T20:27:44.009 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:43 smithi196 ceph-mon[32858]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:44.009 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:43 smithi196 ceph-mon[32858]: Deploying daemon mgr.z on smithi204 2024-05-03T20:27:44.816 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:27:44 smithi204 systemd[1]: Started Ceph mgr.z for 1cd058ee-098b-11ef-bc97-c7b262605968. 2024-05-03T20:27:45.103 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:27:44 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z[35460]: 2024-05-03T20:27:44.816+0000 7fa415a8e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T20:27:45.103 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:27:45 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z[35460]: 2024-05-03T20:27:45.002+0000 7fa415a8e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T20:27:45.103 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:27:45 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z[35460]: 2024-05-03T20:27:45.103+0000 7fa415a8e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T20:27:45.736 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:27:45 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z[35460]: 2024-05-03T20:27:45.412+0000 7fa415a8e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T20:27:45.736 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:27:45 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z[35460]: 2024-05-03T20:27:45.496+0000 7fa415a8e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T20:27:45.754 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-03T20:27:45.755 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-03T20:27:45.755 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2024-05-03T20:27:45.755 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2024-05-03T20:27:45.755 INFO:teuthology.orchestra.run.smithi181.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113888 s, 921 MB/s 2024-05-03T20:27:45.755 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2024-05-03T20:27:46.072 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:45 smithi181 ceph-mon[29801]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:46.073 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:45 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:46.073 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:45 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:46.073 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:45 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:46.073 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:45 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T20:27:46.073 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:45 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T20:27:46.073 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:45 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T20:27:46.073 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:45 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:46.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:45 smithi196 ceph-mon[32858]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:46.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:45 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:46.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:45 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:46.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:45 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:46.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:45 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T20:27:46.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:45 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T20:27:46.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:45 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T20:27:46.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:45 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:46.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:45 smithi204 ceph-mon[33595]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:46.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:45 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:46.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:45 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:46.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:45 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:46.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:45 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T20:27:46.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:45 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T20:27:46.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:45 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T20:27:46.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:45 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:46.440 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_4 2024-05-03T20:27:46.674 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:27:46.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:46 smithi196 ceph-mon[32858]: Deploying daemon mgr.y on smithi196 2024-05-03T20:27:46.857 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:46 smithi196 podman[34239]: 2024-05-03 20:27:46.828581183 +0000 UTC m=+0.123946687 container create fba3b2277bf8851913ed02d81741cb342fe4b0798c7ed1a5afc59ed11cadff47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux ) 2024-05-03T20:27:46.994 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:46 smithi181 ceph-mon[29801]: Deploying daemon mgr.y on smithi196 2024-05-03T20:27:47.040 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:46 smithi204 ceph-mon[33595]: Deploying daemon mgr.y on smithi196 2024-05-03T20:27:47.040 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:27:46 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z[35460]: 2024-05-03T20:27:46.697+0000 7fa415a8e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T20:27:47.040 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:27:46 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z[35460]: 2024-05-03T20:27:46.869+0000 7fa415a8e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T20:27:47.041 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:27:47 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z[35460]: 2024-05-03T20:27:47.041+0000 7fa415a8e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T20:27:47.357 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:47 smithi196 podman[34239]: 2024-05-03 20:27:47.023007969 +0000 UTC m=+0.318373477 container init fba3b2277bf8851913ed02d81741cb342fe4b0798c7ed1a5afc59ed11cadff47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-03T20:27:47.357 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:47 smithi196 podman[34239]: 2024-05-03 20:27:47.025682415 +0000 UTC m=+0.321047923 container start fba3b2277bf8851913ed02d81741cb342fe4b0798c7ed1a5afc59ed11cadff47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430) 2024-05-03T20:27:47.357 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:47 smithi196 ceph-mgr[34258]: -- 172.21.15.196:0/1776428690 <== mon.2 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5596d0e645a0 con 0x5596d0e38c00 2024-05-03T20:27:47.357 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:47 smithi196 bash[34239]: fba3b2277bf8851913ed02d81741cb342fe4b0798c7ed1a5afc59ed11cadff47 2024-05-03T20:27:47.357 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:47 smithi196 systemd[1]: Started Ceph mgr.y for 1cd058ee-098b-11ef-bc97-c7b262605968. 2024-05-03T20:27:47.357 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:47 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y[34254]: 2024-05-03T20:27:47.177+0000 7f9c0cfe7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T20:27:47.613 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:27:47 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z[35460]: 2024-05-03T20:27:47.332+0000 7fa415a8e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T20:27:47.613 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:27:47 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z[35460]: 2024-05-03T20:27:47.428+0000 7fa415a8e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T20:27:47.742 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:47 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y[34254]: 2024-05-03T20:27:47.370+0000 7f9c0cfe7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T20:27:47.742 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:47 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y[34254]: 2024-05-03T20:27:47.479+0000 7f9c0cfe7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T20:27:48.000 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:47 smithi181 ceph-mon[29801]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:48.000 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:47 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:48.000 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:47 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:48.000 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:47 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:48.000 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:47 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:48.000 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:47 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:27:48.077 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:47 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y[34254]: 2024-05-03T20:27:47.810+0000 7f9c0cfe7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T20:27:48.077 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:47 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y[34254]: 2024-05-03T20:27:47.910+0000 7f9c0cfe7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T20:27:48.078 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:47 smithi196 ceph-mon[32858]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:48.078 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:47 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:48.078 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:47 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:48.078 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:47 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:48.078 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:47 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:48.078 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:47 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:27:48.114 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:27:47 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z[35460]: 2024-05-03T20:27:47.642+0000 7fa415a8e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T20:27:48.114 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:27:47 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z[35460]: 2024-05-03T20:27:47.854+0000 7fa415a8e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T20:27:48.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:47 smithi204 ceph-mon[33595]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:48.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:47 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:48.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:47 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:48.115 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:47 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:48.115 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:47 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:48.115 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:47 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:27:48.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.186+0000 7fadc4bb8700 1 -- 172.21.15.181:0/1057278106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc0102400 msgr2=0x7fadc0102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:48.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.186+0000 7fadc4bb8700 1 --2- 172.21.15.181:0/1057278106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc0102400 0x7fadc0102820 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fadb40099c0 tx=0x7fadb4009cd0 comp rx=0 tx=0).stop 2024-05-03T20:27:48.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.187+0000 7fadc4bb8700 1 -- 172.21.15.181:0/1057278106 shutdown_connections 2024-05-03T20:27:48.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.187+0000 7fadc4bb8700 1 --2- 172.21.15.181:0/1057278106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadc0103fa0 0x7fadc010a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:48.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.187+0000 7fadc4bb8700 1 --2- 172.21.15.181:0/1057278106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc0102400 0x7fadc0102820 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:48.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.187+0000 7fadc4bb8700 1 --2- 172.21.15.181:0/1057278106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc0103600 0x7fadc0103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:48.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.187+0000 7fadc4bb8700 1 -- 172.21.15.181:0/1057278106 >> 172.21.15.181:0/1057278106 conn(0x7fadc00fd9c0 msgr2=0x7fadc00ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:27:48.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.187+0000 7fadc4bb8700 1 -- 172.21.15.181:0/1057278106 shutdown_connections 2024-05-03T20:27:48.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.187+0000 7fadc4bb8700 1 -- 172.21.15.181:0/1057278106 wait complete. 2024-05-03T20:27:48.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.187+0000 7fadc4bb8700 1 Processor -- start 2024-05-03T20:27:48.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.187+0000 7fadc4bb8700 1 -- start start 2024-05-03T20:27:48.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.188+0000 7fadc4bb8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadc0102400 0x7fadc006c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:48.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.188+0000 7fadc4bb8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc0103600 0x7fadc019e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:48.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.188+0000 7fadc4bb8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc0103fa0 0x7fadc01a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:48.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.188+0000 7fadc4bb8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fadc010def0 con 0x7fadc0103600 2024-05-03T20:27:48.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.188+0000 7fadc4bb8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fadc010dd70 con 0x7fadc0103fa0 2024-05-03T20:27:48.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.188+0000 7fadc4bb8700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fadc010e070 con 0x7fadc0102400 2024-05-03T20:27:48.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.188+0000 7fadbdd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc0103600 0x7fadc019e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:48.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.188+0000 7fadbdd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc0103600 0x7fadc019e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55874/0 (socket says 172.21.15.181:55874) 2024-05-03T20:27:48.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.188+0000 7fadbe59c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadc0102400 0x7fadc006c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:48.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.188+0000 7fadbdd9b700 1 -- 172.21.15.181:0/2434036556 learned_addr learned my addr 172.21.15.181:0/2434036556 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:27:48.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.188+0000 7fadbed9d700 1 --2- 172.21.15.181:0/2434036556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc0103fa0 0x7fadc01a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:48.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.188+0000 7fadbdd9b700 1 -- 172.21.15.181:0/2434036556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadc0102400 msgr2=0x7fadc006c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:48.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.188+0000 7fadbdd9b700 1 --2- 172.21.15.181:0/2434036556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadc0102400 0x7fadc006c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:48.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.188+0000 7fadbdd9b700 1 -- 172.21.15.181:0/2434036556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc0103fa0 msgr2=0x7fadc01a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:27:48.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.188+0000 7fadbdd9b700 1 --2- 172.21.15.181:0/2434036556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc0103fa0 0x7fadc01a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:27:48.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.188+0000 7fadbdd9b700 1 -- 172.21.15.181:0/2434036556 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc01a12e0 con 0x7fadc0103600 2024-05-03T20:27:48.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.188+0000 7fadbdd9b700 1 --2- 172.21.15.181:0/2434036556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc0103600 0x7fadc019e440 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fadb000e9a0 tx=0x7fadb000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:27:48.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.188+0000 7fadaf7fe700 1 -- 172.21.15.181:0/2434036556 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb000ca10 con 0x7fadc0103600 2024-05-03T20:27:48.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.188+0000 7fadaf7fe700 1 -- 172.21.15.181:0/2434036556 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadb000cbb0 con 0x7fadc0103600 2024-05-03T20:27:48.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.188+0000 7fadc4bb8700 1 -- 172.21.15.181:0/2434036556 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc01a15d0 con 0x7fadc0103600 2024-05-03T20:27:48.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.188+0000 7fadaf7fe700 1 -- 172.21.15.181:0/2434036556 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb0018680 con 0x7fadc0103600 2024-05-03T20:27:48.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.188+0000 7fadc4bb8700 1 -- 172.21.15.181:0/2434036556 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc01a1b10 con 0x7fadc0103600 2024-05-03T20:27:48.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.189+0000 7fadaf7fe700 1 -- 172.21.15.181:0/2434036556 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fadb0018820 con 0x7fadc0103600 2024-05-03T20:27:48.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.189+0000 7fadc4bb8700 1 -- 172.21.15.181:0/2434036556 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadc006a190 con 0x7fadc0103600 2024-05-03T20:27:48.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.189+0000 7fadaf7fe700 1 --2- 172.21.15.181:0/2434036556 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fada803e750 0x7fada8040c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:27:48.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.190+0000 7fadaf7fe700 1 -- 172.21.15.181:0/2434036556 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fadb0052ad0 con 0x7fadc0103600 2024-05-03T20:27:48.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.190+0000 7fadbe59c700 1 --2- 172.21.15.181:0/2434036556 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fada803e750 0x7fada8040c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:27:48.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.190+0000 7fadbe59c700 1 --2- 172.21.15.181:0/2434036556 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fada803e750 0x7fada8040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fadb400b580 tx=0x7fadb4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:27:48.194 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.194+0000 7fadaf7fe700 1 -- 172.21.15.181:0/2434036556 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fadb0020360 con 0x7fadc0103600 2024-05-03T20:27:48.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:48.406+0000 7fadc4bb8700 1 -- 172.21.15.181:0/2434036556 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fadc00638e0 con 0x7fada803e750 2024-05-03T20:27:48.750 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:27:48 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z[35460]: 2024-05-03T20:27:48.485+0000 7fa415a8e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T20:27:48.751 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:27:48 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z[35460]: 2024-05-03T20:27:48.716+0000 7fa415a8e200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T20:27:49.026 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:27:49.026 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:27:49.026 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:49.026 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:49.026 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:49.026 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:49.027 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:49.027 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:49.027 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:27:49.027 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:49.027 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T20:27:49.027 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T20:27:49.027 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:49.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:27:49.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:27:49.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:49.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:49.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:49.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:49.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:49.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:49.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:27:49.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:49.109 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T20:27:49.109 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T20:27:49.109 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:49.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:27:49.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:27:49.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:49.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:49.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:49.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:49.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:49.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:49.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:27:49.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:49.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T20:27:49.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T20:27:49.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:49.488 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:49 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y[34254]: 2024-05-03T20:27:49.150+0000 7f9c0cfe7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T20:27:49.488 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:49 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y[34254]: 2024-05-03T20:27:49.317+0000 7f9c0cfe7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T20:27:49.488 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:49 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y[34254]: 2024-05-03T20:27:49.488+0000 7f9c0cfe7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T20:27:49.613 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:27:49 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z[35460]: 2024-05-03T20:27:49.142+0000 7fa415a8e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T20:27:49.613 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:27:49 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z[35460]: 2024-05-03T20:27:49.238+0000 7fa415a8e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T20:27:49.873 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:49 smithi181 ceph-mon[29801]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:49.873 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:49 smithi181 ceph-mon[29801]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:27:49.873 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:49 smithi181 ceph-mon[29801]: Reconfiguring mgr.x (unknown last config time)... 2024-05-03T20:27:49.873 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:49 smithi181 ceph-mon[29801]: Reconfiguring daemon mgr.x on smithi181 2024-05-03T20:27:49.873 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:49 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:49.986 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:27:49 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z[35460]: 2024-05-03T20:27:49.633+0000 7fa415a8e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T20:27:49.986 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:27:49 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z[35460]: 2024-05-03T20:27:49.986+0000 7fa415a8e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T20:27:49.986 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:49 smithi204 ceph-mon[33595]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:49.986 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:49 smithi204 ceph-mon[33595]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:27:49.986 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:49 smithi204 ceph-mon[33595]: Reconfiguring mgr.x (unknown last config time)... 2024-05-03T20:27:49.987 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:49 smithi204 ceph-mon[33595]: Reconfiguring daemon mgr.x on smithi181 2024-05-03T20:27:49.987 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:49 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:50.070 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:49 smithi196 ceph-mon[32858]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:50.071 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:49 smithi196 ceph-mon[32858]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:27:50.071 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:49 smithi196 ceph-mon[32858]: Reconfiguring mgr.x (unknown last config time)... 2024-05-03T20:27:50.071 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:49 smithi196 ceph-mon[32858]: Reconfiguring daemon mgr.x on smithi181 2024-05-03T20:27:50.071 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:49 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:50.071 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:49 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y[34254]: 2024-05-03T20:27:49.775+0000 7f9c0cfe7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T20:27:50.071 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:49 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y[34254]: 2024-05-03T20:27:49.868+0000 7f9c0cfe7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T20:27:50.071 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:50 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y[34254]: 2024-05-03T20:27:50.071+0000 7f9c0cfe7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T20:27:50.317 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:27:50 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z[35460]: 2024-05-03T20:27:50.078+0000 7fa415a8e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T20:27:50.356 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:50 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y[34254]: 2024-05-03T20:27:50.262+0000 7f9c0cfe7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T20:27:50.613 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:27:50 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z[35460]: 2024-05-03T20:27:50.318+0000 7fa415a8e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T20:27:50.614 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:27:50 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z[35460]: 2024-05-03T20:27:50.450+0000 7fa415a8e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T20:27:51.114 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:27:50 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z[35460]: 2024-05-03T20:27:50.761+0000 7fa415a8e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T20:27:51.114 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:27:50 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z[35460]: 2024-05-03T20:27:50.875+0000 7fa415a8e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T20:27:51.114 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:27:50 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z[35460]: 2024-05-03T20:27:50.966+0000 7fa415a8e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T20:27:51.357 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:50 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y[34254]: 2024-05-03T20:27:50.867+0000 7f9c0cfe7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T20:27:51.357 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:51 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y[34254]: 2024-05-03T20:27:51.087+0000 7f9c0cfe7200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T20:27:51.778 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:27:51 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z[35460]: 2024-05-03T20:27:51.563+0000 7fa415a8e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T20:27:51.778 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:27:51 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z[35460]: 2024-05-03T20:27:51.778+0000 7fa415a8e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T20:27:51.778 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:51 smithi204 ceph-mon[33595]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:51.778 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:51 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:51.778 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:51 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:51.778 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:51 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:27:51.779 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:51 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:51.779 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:51 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:27:51.779 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:51 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:51.857 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:51 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y[34254]: 2024-05-03T20:27:51.508+0000 7f9c0cfe7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T20:27:51.857 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:51 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y[34254]: 2024-05-03T20:27:51.606+0000 7f9c0cfe7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T20:27:51.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:51 smithi196 ceph-mon[32858]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:51.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:51 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:51.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:51 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:51.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:51 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:27:51.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:51 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:51.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:51 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:27:51.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:51 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:51.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:51 smithi181 ceph-mon[29801]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:51.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:51 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:51.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:51 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:51.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:51 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:27:51.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:51 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:51.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:51 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:27:51.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:51 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:27:52.349 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:51 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y[34254]: 2024-05-03T20:27:51.992+0000 7f9c0cfe7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T20:27:52.349 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:52 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y[34254]: 2024-05-03T20:27:52.349+0000 7f9c0cfe7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T20:27:52.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:52.534+0000 7fadaf7fe700 1 -- 172.21.15.181:0/2434036556 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 14) ==== 99938+0+0 (secure 0 0 0) 0x7fadb0018c60 con 0x7fadc0103600 2024-05-03T20:27:52.606 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:52 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y[34254]: 2024-05-03T20:27:52.440+0000 7f9c0cfe7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T20:27:52.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:52 smithi196 ceph-mon[32858]: Standby manager daemon z started 2024-05-03T20:27:52.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:52 smithi196 ceph-mon[32858]: from='mgr.? 172.21.15.204:0/3826209415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-03T20:27:52.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:52 smithi196 ceph-mon[32858]: from='mgr.? 172.21.15.204:0/3826209415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T20:27:52.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:52 smithi196 ceph-mon[32858]: from='mgr.? 172.21.15.204:0/3826209415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-03T20:27:52.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:52 smithi196 ceph-mon[32858]: from='mgr.? 172.21.15.204:0/3826209415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T20:27:52.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:52 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3568343341' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f594d97-10a4-4f77-aebc-c00dbe92a4af"}]: dispatch 2024-05-03T20:27:52.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:52 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3568343341' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f594d97-10a4-4f77-aebc-c00dbe92a4af"}]': finished 2024-05-03T20:27:52.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:52 smithi196 ceph-mon[32858]: osdmap e5: 1 total, 0 up, 1 in 2024-05-03T20:27:52.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:52 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T20:27:52.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:52 smithi181 ceph-mon[29801]: Standby manager daemon z started 2024-05-03T20:27:52.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:52 smithi181 ceph-mon[29801]: from='mgr.? 172.21.15.204:0/3826209415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-03T20:27:52.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:52 smithi181 ceph-mon[29801]: from='mgr.? 172.21.15.204:0/3826209415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T20:27:52.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:52 smithi181 ceph-mon[29801]: from='mgr.? 172.21.15.204:0/3826209415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-03T20:27:52.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:52 smithi181 ceph-mon[29801]: from='mgr.? 172.21.15.204:0/3826209415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T20:27:52.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:52 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3568343341' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f594d97-10a4-4f77-aebc-c00dbe92a4af"}]: dispatch 2024-05-03T20:27:52.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:52 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3568343341' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f594d97-10a4-4f77-aebc-c00dbe92a4af"}]': finished 2024-05-03T20:27:52.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:52 smithi181 ceph-mon[29801]: osdmap e5: 1 total, 0 up, 1 in 2024-05-03T20:27:52.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:52 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T20:27:52.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:52 smithi204 ceph-mon[33595]: Standby manager daemon z started 2024-05-03T20:27:52.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:52 smithi204 ceph-mon[33595]: from='mgr.? 172.21.15.204:0/3826209415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-03T20:27:52.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:52 smithi204 ceph-mon[33595]: from='mgr.? 172.21.15.204:0/3826209415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T20:27:52.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:52 smithi204 ceph-mon[33595]: from='mgr.? 172.21.15.204:0/3826209415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-03T20:27:52.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:52 smithi204 ceph-mon[33595]: from='mgr.? 172.21.15.204:0/3826209415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T20:27:52.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:52 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3568343341' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f594d97-10a4-4f77-aebc-c00dbe92a4af"}]: dispatch 2024-05-03T20:27:52.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:52 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3568343341' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f594d97-10a4-4f77-aebc-c00dbe92a4af"}]': finished 2024-05-03T20:27:52.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:52 smithi204 ceph-mon[33595]: osdmap e5: 1 total, 0 up, 1 in 2024-05-03T20:27:52.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:52 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T20:27:53.107 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:52 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y[34254]: 2024-05-03T20:27:52.683+0000 7f9c0cfe7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T20:27:53.107 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:52 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y[34254]: 2024-05-03T20:27:52.817+0000 7f9c0cfe7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T20:27:53.532 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:53 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y[34254]: 2024-05-03T20:27:53.155+0000 7f9c0cfe7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T20:27:53.532 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:53 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y[34254]: 2024-05-03T20:27:53.271+0000 7f9c0cfe7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T20:27:53.533 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:53 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y[34254]: 2024-05-03T20:27:53.362+0000 7f9c0cfe7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T20:27:53.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:53 smithi196 ceph-mon[32858]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:53.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:53 smithi196 ceph-mon[32858]: mgrmap e14: x(active, since 78s), standbys: z 2024-05-03T20:27:53.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:53 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-03T20:27:53.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:53 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/4018811600' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:27:53.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:53 smithi204 ceph-mon[33595]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:53.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:53 smithi204 ceph-mon[33595]: mgrmap e14: x(active, since 78s), standbys: z 2024-05-03T20:27:53.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:53 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-03T20:27:53.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:53 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/4018811600' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:27:53.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:53 smithi181 ceph-mon[29801]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:53.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:53 smithi181 ceph-mon[29801]: mgrmap e14: x(active, since 78s), standbys: z 2024-05-03T20:27:53.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:53 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-03T20:27:53.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:53 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/4018811600' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:27:54.357 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:53 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y[34254]: 2024-05-03T20:27:53.964+0000 7f9c0cfe7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T20:27:54.357 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:27:54 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y[34254]: 2024-05-03T20:27:54.179+0000 7f9c0cfe7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T20:27:54.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:27:54.546+0000 7fadaf7fe700 1 -- 172.21.15.181:0/2434036556 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7fadb0014230 con 0x7fadc0103600 2024-05-03T20:27:54.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:54 smithi196 ceph-mon[32858]: Standby manager daemon y started 2024-05-03T20:27:54.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:54 smithi196 ceph-mon[32858]: from='mgr.? 172.21.15.196:0/2952174861' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-03T20:27:54.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:54 smithi196 ceph-mon[32858]: from='mgr.? 172.21.15.196:0/2952174861' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T20:27:54.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:54 smithi196 ceph-mon[32858]: from='mgr.? 172.21.15.196:0/2952174861' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-03T20:27:54.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:54 smithi196 ceph-mon[32858]: from='mgr.? 172.21.15.196:0/2952174861' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T20:27:54.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:54 smithi204 ceph-mon[33595]: Standby manager daemon y started 2024-05-03T20:27:54.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:54 smithi204 ceph-mon[33595]: from='mgr.? 172.21.15.196:0/2952174861' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-03T20:27:54.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:54 smithi204 ceph-mon[33595]: from='mgr.? 172.21.15.196:0/2952174861' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T20:27:54.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:54 smithi204 ceph-mon[33595]: from='mgr.? 172.21.15.196:0/2952174861' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-03T20:27:54.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:54 smithi204 ceph-mon[33595]: from='mgr.? 172.21.15.196:0/2952174861' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T20:27:54.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:54 smithi181 ceph-mon[29801]: Standby manager daemon y started 2024-05-03T20:27:54.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:54 smithi181 ceph-mon[29801]: from='mgr.? 172.21.15.196:0/2952174861' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-03T20:27:54.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:54 smithi181 ceph-mon[29801]: from='mgr.? 172.21.15.196:0/2952174861' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T20:27:54.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:54 smithi181 ceph-mon[29801]: from='mgr.? 172.21.15.196:0/2952174861' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-03T20:27:54.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:54 smithi181 ceph-mon[29801]: from='mgr.? 172.21.15.196:0/2952174861' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T20:27:55.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:55 smithi196 ceph-mon[32858]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:55.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:55 smithi196 ceph-mon[32858]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-03T20:27:55.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:55 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-03T20:27:55.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:55 smithi204 ceph-mon[33595]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:55.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:55 smithi204 ceph-mon[33595]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-03T20:27:55.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:55 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-03T20:27:55.889 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:55 smithi181 ceph-mon[29801]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:55.889 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:55 smithi181 ceph-mon[29801]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-03T20:27:55.889 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:55 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-03T20:27:57.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:57 smithi196 ceph-mon[32858]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:57.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:57 smithi204 ceph-mon[33595]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:57.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:57 smithi181 ceph-mon[29801]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:59.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:59 smithi196 ceph-mon[32858]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:59.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:59 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-03T20:27:59.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:27:59 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:59.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:59 smithi204 ceph-mon[33595]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:59.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:59 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-03T20:27:59.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:27:59 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:27:59.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:59 smithi181 ceph-mon[29801]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:27:59.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:59 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-03T20:27:59.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:27:59 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:00.676 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:00 smithi181 ceph-mon[29801]: Deploying daemon osd.0 on smithi181 2024-05-03T20:28:00.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:00 smithi196 ceph-mon[32858]: Deploying daemon osd.0 on smithi181 2024-05-03T20:28:00.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:00 smithi204 ceph-mon[33595]: Deploying daemon osd.0 on smithi181 2024-05-03T20:28:01.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:01 smithi196 ceph-mon[32858]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:28:01.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:01 smithi204 ceph-mon[33595]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:28:01.915 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:01 smithi181 ceph-mon[29801]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:28:03.669 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:03 smithi181 ceph-mon[29801]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:28:03.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:03 smithi196 ceph-mon[32858]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:28:03.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:03 smithi204 ceph-mon[33595]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:28:05.780 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:05 smithi181 ceph-mon[29801]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:28:05.780 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:05 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:05.780 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:05 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:05.780 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:05 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:28:05.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:05 smithi196 ceph-mon[32858]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:28:05.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:05 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:05.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:05 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:05.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:05 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:28:05.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:05 smithi204 ceph-mon[33595]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:28:05.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:05 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:05.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:05 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:05.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:05 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:28:07.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:07 smithi181 ceph-mon[29801]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:28:07.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:07 smithi196 ceph-mon[32858]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:28:07.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:07 smithi204 ceph-mon[33595]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:28:08.470 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 0 on host 'smithi181' 2024-05-03T20:28:08.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:28:08.471+0000 7fadaf7fe700 1 -- 172.21.15.181:0/2434036556 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fadc00638e0 con 0x7fada803e750 2024-05-03T20:28:08.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:28:08.474+0000 7fadc4bb8700 1 -- 172.21.15.181:0/2434036556 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fada803e750 msgr2=0x7fada8040c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:28:08.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:28:08.474+0000 7fadc4bb8700 1 --2- 172.21.15.181:0/2434036556 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fada803e750 0x7fada8040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fadb400b580 tx=0x7fadb4005fb0 comp rx=0 tx=0).stop 2024-05-03T20:28:08.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:28:08.474+0000 7fadc4bb8700 1 -- 172.21.15.181:0/2434036556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc0103600 msgr2=0x7fadc019e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:28:08.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:28:08.474+0000 7fadc4bb8700 1 --2- 172.21.15.181:0/2434036556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc0103600 0x7fadc019e440 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fadb000e9a0 tx=0x7fadb000ecb0 comp rx=0 tx=0).stop 2024-05-03T20:28:08.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:28:08.474+0000 7fadc4bb8700 1 -- 172.21.15.181:0/2434036556 shutdown_connections 2024-05-03T20:28:08.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:28:08.474+0000 7fadc4bb8700 1 --2- 172.21.15.181:0/2434036556 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fada803e750 0x7fada8040c10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:28:08.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:28:08.474+0000 7fadc4bb8700 1 --2- 172.21.15.181:0/2434036556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc0103fa0 0x7fadc01a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:28:08.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:28:08.474+0000 7fadc4bb8700 1 --2- 172.21.15.181:0/2434036556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadc0102400 0x7fadc006c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:28:08.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:28:08.474+0000 7fadc4bb8700 1 --2- 172.21.15.181:0/2434036556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc0103600 0x7fadc019e440 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:28:08.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:28:08.474+0000 7fadc4bb8700 1 -- 172.21.15.181:0/2434036556 >> 172.21.15.181:0/2434036556 conn(0x7fadc00fd9c0 msgr2=0x7fadc0106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:28:08.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:28:08.474+0000 7fadc4bb8700 1 -- 172.21.15.181:0/2434036556 shutdown_connections 2024-05-03T20:28:08.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:28:08.474+0000 7fadc4bb8700 1 -- 172.21.15.181:0/2434036556 wait complete. 2024-05-03T20:28:08.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:08 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:08.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:08 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:08.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:08 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:08.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:08 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:28:08.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:08 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:08.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:08 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:28:08.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:08 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:08.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:08 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:08.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:08 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:08.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:08 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:08.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:08 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:08.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:08 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:28:08.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:08 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:08.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:08 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:28:08.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:08 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:08.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:08 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:08.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:08 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:08.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:08 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:08.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:08 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:08.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:08 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:28:08.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:08 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:08.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:08 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:28:08.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:08 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:08.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:08 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:09.097 DEBUG:teuthology.orchestra.run.smithi181:osd.0> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.0.service 2024-05-03T20:28:09.100 INFO:tasks.cephadm:Deploying osd.1 on smithi196 with /dev/vg_nvme/lv_4... 2024-05-03T20:28:09.100 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-03T20:28:09.343 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T20:28:09.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:09 smithi181 ceph-mon[29801]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:28:09.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:09 smithi181 ceph-mon[29801]: from='osd.0 [v2:172.21.15.181:6802/1683973278,v1:172.21.15.181:6803/1683973278]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-03T20:28:09.734 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:09 smithi196 ceph-mon[32858]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:28:09.734 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:09 smithi196 ceph-mon[32858]: from='osd.0 [v2:172.21.15.181:6802/1683973278,v1:172.21.15.181:6803/1683973278]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-03T20:28:10.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:09 smithi204 ceph-mon[33595]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:28:10.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:09 smithi204 ceph-mon[33595]: from='osd.0 [v2:172.21.15.181:6802/1683973278,v1:172.21.15.181:6803/1683973278]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-03T20:28:10.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:10 smithi196 ceph-mon[32858]: from='osd.0 [v2:172.21.15.181:6802/1683973278,v1:172.21.15.181:6803/1683973278]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-03T20:28:10.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:10 smithi196 ceph-mon[32858]: osdmap e6: 1 total, 0 up, 1 in 2024-05-03T20:28:10.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:10 smithi196 ceph-mon[32858]: from='osd.0 [v2:172.21.15.181:6802/1683973278,v1:172.21.15.181:6803/1683973278]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-03T20:28:10.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:10 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T20:28:10.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:10 smithi181 ceph-mon[29801]: from='osd.0 [v2:172.21.15.181:6802/1683973278,v1:172.21.15.181:6803/1683973278]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-03T20:28:10.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:10 smithi181 ceph-mon[29801]: osdmap e6: 1 total, 0 up, 1 in 2024-05-03T20:28:10.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:10 smithi181 ceph-mon[29801]: from='osd.0 [v2:172.21.15.181:6802/1683973278,v1:172.21.15.181:6803/1683973278]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-03T20:28:10.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:10 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T20:28:11.084 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-03T20:28:11.085 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-03T20:28:11.085 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-05-03T20:28:11.085 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-05-03T20:28:11.085 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112978 s, 928 MB/s 2024-05-03T20:28:11.085 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-05-03T20:28:11.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:10 smithi204 ceph-mon[33595]: from='osd.0 [v2:172.21.15.181:6802/1683973278,v1:172.21.15.181:6803/1683973278]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-03T20:28:11.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:10 smithi204 ceph-mon[33595]: osdmap e6: 1 total, 0 up, 1 in 2024-05-03T20:28:11.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:10 smithi204 ceph-mon[33595]: from='osd.0 [v2:172.21.15.181:6802/1683973278,v1:172.21.15.181:6803/1683973278]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-03T20:28:11.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:10 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T20:28:11.652 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:11 smithi196 ceph-mon[32858]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:28:11.652 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:11 smithi196 ceph-mon[32858]: from='osd.0 [v2:172.21.15.181:6802/1683973278,v1:172.21.15.181:6803/1683973278]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-03T20:28:11.652 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:11 smithi196 ceph-mon[32858]: osdmap e7: 1 total, 0 up, 1 in 2024-05-03T20:28:11.652 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:11 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T20:28:11.652 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:11 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T20:28:11.652 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:11 smithi196 ceph-mon[32858]: from='osd.0 [v2:172.21.15.181:6802/1683973278,v1:172.21.15.181:6803/1683973278]' entity='osd.0' 2024-05-03T20:28:11.679 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_4 2024-05-03T20:28:11.701 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:11 smithi181 ceph-mon[29801]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:28:11.701 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:11 smithi181 ceph-mon[29801]: from='osd.0 [v2:172.21.15.181:6802/1683973278,v1:172.21.15.181:6803/1683973278]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-03T20:28:11.701 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:11 smithi181 ceph-mon[29801]: osdmap e7: 1 total, 0 up, 1 in 2024-05-03T20:28:11.701 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:11 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T20:28:11.701 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:11 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T20:28:11.701 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:11 smithi181 ceph-mon[29801]: from='osd.0 [v2:172.21.15.181:6802/1683973278,v1:172.21.15.181:6803/1683973278]' entity='osd.0' 2024-05-03T20:28:11.701 INFO:journalctl@ceph.osd.0.smithi181.stdout:May 03 20:28:11 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-0[45040]: 2024-05-03T20:28:11.532+0000 7fbaf1ad6700 -1 osd.0 0 waiting for initial osdmap 2024-05-03T20:28:11.701 INFO:journalctl@ceph.osd.0.smithi181.stdout:May 03 20:28:11 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-0[45040]: 2024-05-03T20:28:11.702+0000 7fbae98c1700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T20:28:11.924 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T20:28:12.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:11 smithi204 ceph-mon[33595]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:28:12.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:11 smithi204 ceph-mon[33595]: from='osd.0 [v2:172.21.15.181:6802/1683973278,v1:172.21.15.181:6803/1683973278]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-03T20:28:12.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:11 smithi204 ceph-mon[33595]: osdmap e7: 1 total, 0 up, 1 in 2024-05-03T20:28:12.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:11 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T20:28:12.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:11 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T20:28:12.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:11 smithi204 ceph-mon[33595]: from='osd.0 [v2:172.21.15.181:6802/1683973278,v1:172.21.15.181:6803/1683973278]' entity='osd.0' 2024-05-03T20:28:12.915 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:12 smithi196 ceph-mon[32858]: purged_snaps scrub starts 2024-05-03T20:28:12.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:12 smithi196 ceph-mon[32858]: purged_snaps scrub ok 2024-05-03T20:28:12.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T20:28:12.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:12.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:12.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:28:12.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:12.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:12.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:28:12.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:12.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:12 smithi181 ceph-mon[29801]: purged_snaps scrub starts 2024-05-03T20:28:12.998 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:12 smithi181 ceph-mon[29801]: purged_snaps scrub ok 2024-05-03T20:28:12.999 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T20:28:12.999 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:12.999 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:12.999 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:28:12.999 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:12.999 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:12.999 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:28:12.999 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:13.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:12 smithi204 ceph-mon[33595]: purged_snaps scrub starts 2024-05-03T20:28:13.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:12 smithi204 ceph-mon[33595]: purged_snaps scrub ok 2024-05-03T20:28:13.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T20:28:13.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:13.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:13.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:28:13.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:13.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:13.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:28:13.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:13.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.423+0000 7ff519db5700 1 -- 172.21.15.196:0/2835820706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff514103500 msgr2=0x7ff514103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:28:13.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.423+0000 7ff519db5700 1 --2- 172.21.15.196:0/2835820706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff514103500 0x7ff514103920 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff5040099c0 tx=0x7ff504009cd0 comp rx=0 tx=0).stop 2024-05-03T20:28:13.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.423+0000 7ff519db5700 1 -- 172.21.15.196:0/2835820706 shutdown_connections 2024-05-03T20:28:13.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.423+0000 7ff519db5700 1 --2- 172.21.15.196:0/2835820706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5141050a0 0x7ff51410b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:28:13.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.423+0000 7ff519db5700 1 --2- 172.21.15.196:0/2835820706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff514103500 0x7ff514103920 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:28:13.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.423+0000 7ff519db5700 1 --2- 172.21.15.196:0/2835820706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff514104700 0x7ff514104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:28:13.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.423+0000 7ff519db5700 1 -- 172.21.15.196:0/2835820706 >> 172.21.15.196:0/2835820706 conn(0x7ff5140feaa0 msgr2=0x7ff514100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:28:13.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.423+0000 7ff519db5700 1 -- 172.21.15.196:0/2835820706 shutdown_connections 2024-05-03T20:28:13.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.423+0000 7ff519db5700 1 -- 172.21.15.196:0/2835820706 wait complete. 2024-05-03T20:28:13.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.423+0000 7ff519db5700 1 Processor -- start 2024-05-03T20:28:13.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.424+0000 7ff519db5700 1 -- start start 2024-05-03T20:28:13.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.424+0000 7ff519db5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff514103500 0x7ff5141996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:28:13.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.424+0000 7ff519db5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff514104700 0x7ff514199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:28:13.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.424+0000 7ff519db5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5141050a0 0x7ff5141a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:28:13.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.424+0000 7ff519db5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff51410efa0 con 0x7ff514103500 2024-05-03T20:28:13.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.424+0000 7ff519db5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff51410ee20 con 0x7ff5141050a0 2024-05-03T20:28:13.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.424+0000 7ff519db5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff51410f120 con 0x7ff514104700 2024-05-03T20:28:13.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.424+0000 7ff513fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5141050a0 0x7ff5141a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:28:13.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.424+0000 7ff513fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5141050a0 0x7ff5141a0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:56008/0 (socket says 172.21.15.196:56008) 2024-05-03T20:28:13.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.424+0000 7ff513fff700 1 -- 172.21.15.196:0/4246993876 learned_addr learned my addr 172.21.15.196:0/4246993876 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-05-03T20:28:13.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.424+0000 7ff5137fe700 1 --2- 172.21.15.196:0/4246993876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff514103500 0x7ff5141996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:28:13.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.424+0000 7ff512ffd700 1 --2- 172.21.15.196:0/4246993876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff514104700 0x7ff514199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:28:13.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.424+0000 7ff513fff700 1 -- 172.21.15.196:0/4246993876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff514104700 msgr2=0x7ff514199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:28:13.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.424+0000 7ff513fff700 1 --2- 172.21.15.196:0/4246993876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff514104700 0x7ff514199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:28:13.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.424+0000 7ff513fff700 1 -- 172.21.15.196:0/4246993876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff514103500 msgr2=0x7ff5141996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:28:13.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.424+0000 7ff513fff700 1 --2- 172.21.15.196:0/4246993876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff514103500 0x7ff5141996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:28:13.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.424+0000 7ff513fff700 1 -- 172.21.15.196:0/4246993876 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5141a1320 con 0x7ff5141050a0 2024-05-03T20:28:13.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.424+0000 7ff513fff700 1 --2- 172.21.15.196:0/4246993876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5141050a0 0x7ff5141a0c80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff50c00ef30 tx=0x7ff50c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:28:13.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.425+0000 7ff510ff9700 1 -- 172.21.15.196:0/4246993876 <== mon.2 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c0092e0 con 0x7ff5141050a0 2024-05-03T20:28:13.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.425+0000 7ff519db5700 1 -- 172.21.15.196:0/4246993876 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5141a15b0 con 0x7ff5141050a0 2024-05-03T20:28:13.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.425+0000 7ff510ff9700 1 -- 172.21.15.196:0/4246993876 <== mon.2 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff50c004bb0 con 0x7ff5141050a0 2024-05-03T20:28:13.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.425+0000 7ff510ff9700 1 -- 172.21.15.196:0/4246993876 <== mon.2 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c004020 con 0x7ff5141050a0 2024-05-03T20:28:13.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.425+0000 7ff519db5700 1 -- 172.21.15.196:0/4246993876 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5141a1aa0 con 0x7ff5141050a0 2024-05-03T20:28:13.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.426+0000 7ff519db5700 1 -- 172.21.15.196:0/4246993876 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff51406a190 con 0x7ff5141050a0 2024-05-03T20:28:13.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.427+0000 7ff510ff9700 1 -- 172.21.15.196:0/4246993876 <== mon.2 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7ff50c01b030 con 0x7ff5141050a0 2024-05-03T20:28:13.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.427+0000 7ff510ff9700 1 --2- 172.21.15.196:0/4246993876 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7ff4fc0b27f0 0x7ff4fc0b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:28:13.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.428+0000 7ff510ff9700 1 -- 172.21.15.196:0/4246993876 <== mon.2 v2:172.21.15.196:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7ff50c0e01d0 con 0x7ff5141050a0 2024-05-03T20:28:13.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.428+0000 7ff5137fe700 1 --2- 172.21.15.196:0/4246993876 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7ff4fc0b27f0 0x7ff4fc0b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:28:13.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.428+0000 7ff5137fe700 1 --2- 172.21.15.196:0/4246993876 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7ff4fc0b27f0 0x7ff4fc0b4cb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff50400b580 tx=0x7ff504005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:28:13.431 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.432+0000 7ff510ff9700 1 -- 172.21.15.196:0/4246993876 <== mon.2 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff50c022020 con 0x7ff5141050a0 2024-05-03T20:28:13.643 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:13.644+0000 7ff519db5700 1 -- 172.21.15.196:0/4246993876 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff5140638e0 con 0x7ff4fc0b27f0 2024-05-03T20:28:13.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:13 smithi181 ceph-mon[29801]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:28:13.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:13 smithi181 ceph-mon[29801]: Detected new or changed devices on smithi181 2024-05-03T20:28:13.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:13 smithi181 ceph-mon[29801]: Adjusting osd_memory_target on smithi181 to 17088M 2024-05-03T20:28:13.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:13 smithi181 ceph-mon[29801]: osd.0 [v2:172.21.15.181:6802/1683973278,v1:172.21.15.181:6803/1683973278] boot 2024-05-03T20:28:13.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:13 smithi181 ceph-mon[29801]: osdmap e8: 1 total, 1 up, 1 in 2024-05-03T20:28:13.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:13 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T20:28:14.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:13 smithi196 ceph-mon[32858]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:28:14.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:13 smithi196 ceph-mon[32858]: Detected new or changed devices on smithi181 2024-05-03T20:28:14.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:13 smithi196 ceph-mon[32858]: Adjusting osd_memory_target on smithi181 to 17088M 2024-05-03T20:28:14.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:13 smithi196 ceph-mon[32858]: osd.0 [v2:172.21.15.181:6802/1683973278,v1:172.21.15.181:6803/1683973278] boot 2024-05-03T20:28:14.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:13 smithi196 ceph-mon[32858]: osdmap e8: 1 total, 1 up, 1 in 2024-05-03T20:28:14.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:13 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T20:28:14.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:13 smithi204 ceph-mon[33595]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T20:28:14.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:13 smithi204 ceph-mon[33595]: Detected new or changed devices on smithi181 2024-05-03T20:28:14.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:13 smithi204 ceph-mon[33595]: Adjusting osd_memory_target on smithi181 to 17088M 2024-05-03T20:28:14.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:13 smithi204 ceph-mon[33595]: osd.0 [v2:172.21.15.181:6802/1683973278,v1:172.21.15.181:6803/1683973278] boot 2024-05-03T20:28:14.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:13 smithi204 ceph-mon[33595]: osdmap e8: 1 total, 1 up, 1 in 2024-05-03T20:28:14.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:13 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T20:28:14.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:14 smithi196 ceph-mon[32858]: osdmap e9: 1 total, 1 up, 1 in 2024-05-03T20:28:14.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:14 smithi196 ceph-mon[32858]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:28:14.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:14 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:28:14.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:14 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:28:14.934 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:14 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:14.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:14 smithi181 ceph-mon[29801]: osdmap e9: 1 total, 1 up, 1 in 2024-05-03T20:28:14.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:14 smithi181 ceph-mon[29801]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:28:14.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:14 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:28:14.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:14 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:28:14.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:14 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:15.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:14 smithi204 ceph-mon[33595]: osdmap e9: 1 total, 1 up, 1 in 2024-05-03T20:28:15.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:14 smithi204 ceph-mon[33595]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:28:15.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:14 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:28:15.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:14 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:28:15.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:14 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:15.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:15 smithi196 ceph-mon[32858]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:15.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:15 smithi181 ceph-mon[29801]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:16.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:15 smithi204 ceph-mon[33595]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:16.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:16 smithi196 ceph-mon[32858]: from='client.? 172.21.15.196:0/2151524680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9726d641-8930-4b45-9192-6ee2b6bfcdcc"}]: dispatch 2024-05-03T20:28:16.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:16 smithi196 ceph-mon[32858]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9726d641-8930-4b45-9192-6ee2b6bfcdcc"}]: dispatch 2024-05-03T20:28:16.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:16 smithi196 ceph-mon[32858]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9726d641-8930-4b45-9192-6ee2b6bfcdcc"}]': finished 2024-05-03T20:28:16.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:16 smithi196 ceph-mon[32858]: osdmap e10: 2 total, 1 up, 2 in 2024-05-03T20:28:16.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:16 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T20:28:16.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:16 smithi181 ceph-mon[29801]: from='client.? 172.21.15.196:0/2151524680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9726d641-8930-4b45-9192-6ee2b6bfcdcc"}]: dispatch 2024-05-03T20:28:16.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:16 smithi181 ceph-mon[29801]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9726d641-8930-4b45-9192-6ee2b6bfcdcc"}]: dispatch 2024-05-03T20:28:16.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:16 smithi181 ceph-mon[29801]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9726d641-8930-4b45-9192-6ee2b6bfcdcc"}]': finished 2024-05-03T20:28:16.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:16 smithi181 ceph-mon[29801]: osdmap e10: 2 total, 1 up, 2 in 2024-05-03T20:28:16.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:16 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T20:28:17.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:16 smithi204 ceph-mon[33595]: from='client.? 172.21.15.196:0/2151524680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9726d641-8930-4b45-9192-6ee2b6bfcdcc"}]: dispatch 2024-05-03T20:28:17.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:16 smithi204 ceph-mon[33595]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9726d641-8930-4b45-9192-6ee2b6bfcdcc"}]: dispatch 2024-05-03T20:28:17.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:16 smithi204 ceph-mon[33595]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9726d641-8930-4b45-9192-6ee2b6bfcdcc"}]': finished 2024-05-03T20:28:17.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:16 smithi204 ceph-mon[33595]: osdmap e10: 2 total, 1 up, 2 in 2024-05-03T20:28:17.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:16 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T20:28:17.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:17 smithi181 ceph-mon[29801]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:17.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:17 smithi181 ceph-mon[29801]: from='client.? 172.21.15.196:0/4142022818' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:28:18.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:17 smithi196 ceph-mon[32858]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:18.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:17 smithi196 ceph-mon[32858]: from='client.? 172.21.15.196:0/4142022818' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:28:18.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:17 smithi204 ceph-mon[33595]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:18.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:17 smithi204 ceph-mon[33595]: from='client.? 172.21.15.196:0/4142022818' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:28:19.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:19 smithi181 ceph-mon[29801]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:20.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:19 smithi196 ceph-mon[32858]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:20.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:19 smithi204 ceph-mon[33595]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:21.720 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:21 smithi196 ceph-mon[32858]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:21.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:21 smithi181 ceph-mon[29801]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:22.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:21 smithi204 ceph-mon[33595]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:23.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:23 smithi196 ceph-mon[32858]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:23.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:23 smithi181 ceph-mon[29801]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:24.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:23 smithi204 ceph-mon[33595]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:24.952 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:24 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-03T20:28:24.952 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:24 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:24.952 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:24 smithi196 ceph-mon[32858]: Deploying daemon osd.1 on smithi196 2024-05-03T20:28:24.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:24 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-03T20:28:24.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:24 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:24.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:24 smithi181 ceph-mon[29801]: Deploying daemon osd.1 on smithi196 2024-05-03T20:28:25.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:24 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-03T20:28:25.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:24 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:25.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:24 smithi204 ceph-mon[33595]: Deploying daemon osd.1 on smithi196 2024-05-03T20:28:25.833 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:25 smithi196 ceph-mon[32858]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:25.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:25 smithi181 ceph-mon[29801]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:26.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:25 smithi204 ceph-mon[33595]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:27.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:27 smithi181 ceph-mon[29801]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:28.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:27 smithi196 ceph-mon[32858]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:28.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:27 smithi204 ceph-mon[33595]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:29.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:29 smithi181 ceph-mon[29801]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:29.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:29 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:29.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:29 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:29.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:29 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:28:30.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:29 smithi196 ceph-mon[32858]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:30.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:29 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:30.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:29 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:30.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:29 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:28:30.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:29 smithi204 ceph-mon[33595]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:30.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:29 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:30.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:29 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:30.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:29 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:28:31.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:31 smithi181 ceph-mon[29801]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:32.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:31 smithi196 ceph-mon[32858]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:32.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:31 smithi204 ceph-mon[33595]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:32.205 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:32.206+0000 7ff510ff9700 1 -- 172.21.15.196:0/4246993876 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff5140638e0 con 0x7ff4fc0b27f0 2024-05-03T20:28:32.205 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 1 on host 'smithi196' 2024-05-03T20:28:32.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:32.208+0000 7ff519db5700 1 -- 172.21.15.196:0/4246993876 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7ff4fc0b27f0 msgr2=0x7ff4fc0b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:28:32.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:32.208+0000 7ff519db5700 1 --2- 172.21.15.196:0/4246993876 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7ff4fc0b27f0 0x7ff4fc0b4cb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff50400b580 tx=0x7ff504005fb0 comp rx=0 tx=0).stop 2024-05-03T20:28:32.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:32.208+0000 7ff519db5700 1 -- 172.21.15.196:0/4246993876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5141050a0 msgr2=0x7ff5141a0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:28:32.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:32.208+0000 7ff519db5700 1 --2- 172.21.15.196:0/4246993876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5141050a0 0x7ff5141a0c80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff50c00ef30 tx=0x7ff50c00c3a0 comp rx=0 tx=0).stop 2024-05-03T20:28:32.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:32.208+0000 7ff519db5700 1 -- 172.21.15.196:0/4246993876 shutdown_connections 2024-05-03T20:28:32.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:32.208+0000 7ff519db5700 1 --2- 172.21.15.196:0/4246993876 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7ff4fc0b27f0 0x7ff4fc0b4cb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:28:32.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:32.208+0000 7ff519db5700 1 --2- 172.21.15.196:0/4246993876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5141050a0 0x7ff5141a0c80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:28:32.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:32.208+0000 7ff519db5700 1 --2- 172.21.15.196:0/4246993876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff514103500 0x7ff5141996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:28:32.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:32.208+0000 7ff519db5700 1 --2- 172.21.15.196:0/4246993876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff514104700 0x7ff514199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:28:32.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:32.208+0000 7ff519db5700 1 -- 172.21.15.196:0/4246993876 >> 172.21.15.196:0/4246993876 conn(0x7ff5140feaa0 msgr2=0x7ff514107ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:28:32.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:32.208+0000 7ff519db5700 1 -- 172.21.15.196:0/4246993876 shutdown_connections 2024-05-03T20:28:32.208 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:28:32.208+0000 7ff519db5700 1 -- 172.21.15.196:0/4246993876 wait complete. 2024-05-03T20:28:32.721 DEBUG:teuthology.orchestra.run.smithi196:osd.1> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.1.service 2024-05-03T20:28:32.724 INFO:tasks.cephadm:Deploying osd.2 on smithi204 with /dev/vg_nvme/lv_4... 2024-05-03T20:28:32.724 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-03T20:28:32.973 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T20:28:33.095 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:32 smithi204 ceph-mon[33595]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:33.095 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:32 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:33.095 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:32 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:33.095 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:32 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:33.095 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:32 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:28:33.095 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:32 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:33.096 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:32 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:33.096 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:32 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:33.096 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:32 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:28:33.096 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:32 smithi204 ceph-mon[33595]: from='osd.1 [v2:172.21.15.196:6800/2136873375,v1:172.21.15.196:6801/2136873375]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T20:28:33.096 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:32 smithi204 ceph-mon[33595]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T20:28:33.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:32 smithi196 ceph-mon[32858]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:33.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:32 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:33.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:32 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:33.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:32 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:33.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:32 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:28:33.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:32 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:33.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:32 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:33.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:32 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:33.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:32 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:28:33.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:32 smithi196 ceph-mon[32858]: from='osd.1 [v2:172.21.15.196:6800/2136873375,v1:172.21.15.196:6801/2136873375]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T20:28:33.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:32 smithi196 ceph-mon[32858]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T20:28:33.108 INFO:journalctl@ceph.osd.1.smithi196.stdout:May 03 20:28:32 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-1[37481]: 2024-05-03T20:28:32.840+0000 7ff0844017c0 -1 osd.1 0 log_to_monitors true 2024-05-03T20:28:33.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:32 smithi181 ceph-mon[29801]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:33.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:32 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:33.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:32 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:33.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:32 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:33.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:32 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:28:33.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:32 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:33.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:32 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:33.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:32 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:33.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:32 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:28:33.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:32 smithi181 ceph-mon[29801]: from='osd.1 [v2:172.21.15.196:6800/2136873375,v1:172.21.15.196:6801/2136873375]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T20:28:33.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:32 smithi181 ceph-mon[29801]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T20:28:34.812 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-03T20:28:34.812 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-03T20:28:34.812 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-05-03T20:28:34.812 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-05-03T20:28:34.812 INFO:teuthology.orchestra.run.smithi204.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112601 s, 931 MB/s 2024-05-03T20:28:34.812 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-05-03T20:28:34.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:34 smithi181 ceph-mon[29801]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-03T20:28:34.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:34 smithi181 ceph-mon[29801]: osdmap e11: 2 total, 1 up, 2 in 2024-05-03T20:28:34.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:34 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T20:28:34.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:34 smithi181 ceph-mon[29801]: from='osd.1 [v2:172.21.15.196:6800/2136873375,v1:172.21.15.196:6801/2136873375]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-03T20:28:34.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:34 smithi181 ceph-mon[29801]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-03T20:28:35.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:34 smithi196 ceph-mon[32858]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-03T20:28:35.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:34 smithi196 ceph-mon[32858]: osdmap e11: 2 total, 1 up, 2 in 2024-05-03T20:28:35.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:34 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T20:28:35.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:34 smithi196 ceph-mon[32858]: from='osd.1 [v2:172.21.15.196:6800/2136873375,v1:172.21.15.196:6801/2136873375]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-03T20:28:35.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:34 smithi196 ceph-mon[32858]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-03T20:28:35.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:34 smithi204 ceph-mon[33595]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-03T20:28:35.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:34 smithi204 ceph-mon[33595]: osdmap e11: 2 total, 1 up, 2 in 2024-05-03T20:28:35.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:34 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T20:28:35.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:34 smithi204 ceph-mon[33595]: from='osd.1 [v2:172.21.15.196:6800/2136873375,v1:172.21.15.196:6801/2136873375]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-03T20:28:35.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:34 smithi204 ceph-mon[33595]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-03T20:28:35.409 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_4 2024-05-03T20:28:35.660 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T20:28:35.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:35 smithi181 ceph-mon[29801]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:35.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:35 smithi181 ceph-mon[29801]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-03T20:28:35.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:35 smithi181 ceph-mon[29801]: osdmap e12: 2 total, 1 up, 2 in 2024-05-03T20:28:35.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:35 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T20:28:35.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:35 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T20:28:35.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:35 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:35.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:35 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:35.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:35 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:28:35.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:35 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:35.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:35 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:35.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:35 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:28:35.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:35 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:35.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:35 smithi181 ceph-mon[29801]: from='osd.1 ' entity='osd.1' 2024-05-03T20:28:36.026 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:35 smithi204 ceph-mon[33595]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:36.026 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:35 smithi204 ceph-mon[33595]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-03T20:28:36.026 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:35 smithi204 ceph-mon[33595]: osdmap e12: 2 total, 1 up, 2 in 2024-05-03T20:28:36.026 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:35 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T20:28:36.026 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:35 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T20:28:36.027 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:35 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:36.027 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:35 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:36.027 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:35 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:28:36.027 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:35 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:36.027 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:35 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:36.027 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:35 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:28:36.027 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:35 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:36.027 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:35 smithi204 ceph-mon[33595]: from='osd.1 ' entity='osd.1' 2024-05-03T20:28:36.107 INFO:journalctl@ceph.osd.1.smithi196.stdout:May 03 20:28:35 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-1[37481]: 2024-05-03T20:28:35.641+0000 7ff079a65700 -1 osd.1 0 waiting for initial osdmap 2024-05-03T20:28:36.107 INFO:journalctl@ceph.osd.1.smithi196.stdout:May 03 20:28:35 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-1[37481]: 2024-05-03T20:28:35.845+0000 7ff071850700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T20:28:36.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:35 smithi196 ceph-mon[32858]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:36.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:35 smithi196 ceph-mon[32858]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-03T20:28:36.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:35 smithi196 ceph-mon[32858]: osdmap e12: 2 total, 1 up, 2 in 2024-05-03T20:28:36.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:35 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T20:28:36.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:35 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T20:28:36.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:35 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:36.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:35 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:36.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:35 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:28:36.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:35 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:36.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:35 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:36.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:35 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:28:36.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:35 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:36.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:35 smithi196 ceph-mon[32858]: from='osd.1 ' entity='osd.1' 2024-05-03T20:28:36.729 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:36 smithi204 ceph-mon[33595]: purged_snaps scrub starts 2024-05-03T20:28:36.729 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:36 smithi204 ceph-mon[33595]: purged_snaps scrub ok 2024-05-03T20:28:36.729 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:36 smithi204 ceph-mon[33595]: Detected new or changed devices on smithi196 2024-05-03T20:28:36.729 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:36 smithi204 ceph-mon[33595]: Adjusting osd_memory_target on smithi196 to 17088M 2024-05-03T20:28:36.729 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T20:28:36.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:36 smithi181 ceph-mon[29801]: purged_snaps scrub starts 2024-05-03T20:28:36.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:36 smithi181 ceph-mon[29801]: purged_snaps scrub ok 2024-05-03T20:28:36.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:36 smithi181 ceph-mon[29801]: Detected new or changed devices on smithi196 2024-05-03T20:28:36.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:36 smithi181 ceph-mon[29801]: Adjusting osd_memory_target on smithi196 to 17088M 2024-05-03T20:28:36.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T20:28:37.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:36 smithi196 ceph-mon[32858]: purged_snaps scrub starts 2024-05-03T20:28:37.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:36 smithi196 ceph-mon[32858]: purged_snaps scrub ok 2024-05-03T20:28:37.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:36 smithi196 ceph-mon[32858]: Detected new or changed devices on smithi196 2024-05-03T20:28:37.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:36 smithi196 ceph-mon[32858]: Adjusting osd_memory_target on smithi196 to 17088M 2024-05-03T20:28:37.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T20:28:37.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.220+0000 7f52f4476700 1 -- 172.21.15.204:0/1199021580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52ec103600 msgr2=0x7f52ec103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:28:37.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.220+0000 7f52f4476700 1 --2- 172.21.15.204:0/1199021580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52ec103600 0x7f52ec103a60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009990 tx=0x7f52dc009ca0 comp rx=0 tx=0).stop 2024-05-03T20:28:37.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.220+0000 7f52f4476700 1 -- 172.21.15.204:0/1199021580 shutdown_connections 2024-05-03T20:28:37.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.220+0000 7f52f4476700 1 --2- 172.21.15.204:0/1199021580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52ec103fa0 0x7f52ec10a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:28:37.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.220+0000 7f52f4476700 1 --2- 172.21.15.204:0/1199021580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52ec102400 0x7f52ec102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:28:37.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.220+0000 7f52f4476700 1 --2- 172.21.15.204:0/1199021580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52ec103600 0x7f52ec103a60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:28:37.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.220+0000 7f52f4476700 1 -- 172.21.15.204:0/1199021580 >> 172.21.15.204:0/1199021580 conn(0x7f52ec0fd9c0 msgr2=0x7f52ec0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:28:37.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.220+0000 7f52f4476700 1 -- 172.21.15.204:0/1199021580 shutdown_connections 2024-05-03T20:28:37.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.220+0000 7f52f4476700 1 -- 172.21.15.204:0/1199021580 wait complete. 2024-05-03T20:28:37.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.220+0000 7f52f4476700 1 Processor -- start 2024-05-03T20:28:37.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.220+0000 7f52f4476700 1 -- start start 2024-05-03T20:28:37.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.220+0000 7f52f4476700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52ec102400 0x7f52ec06c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:28:37.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.221+0000 7f52f4476700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52ec103600 0x7f52ec19e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:28:37.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.221+0000 7f52f4476700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52ec103fa0 0x7f52ec1a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:28:37.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.221+0000 7f52f4476700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f52ec10de80 con 0x7f52ec103600 2024-05-03T20:28:37.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.221+0000 7f52f4476700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f52ec10dd00 con 0x7f52ec103fa0 2024-05-03T20:28:37.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.221+0000 7f52f4476700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f52ec10e000 con 0x7f52ec102400 2024-05-03T20:28:37.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.221+0000 7f52f2212700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52ec102400 0x7f52ec06c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:28:37.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.221+0000 7f52f2212700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52ec102400 0x7f52ec06c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:41808/0 (socket says 172.21.15.204:41808) 2024-05-03T20:28:37.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.221+0000 7f52f2212700 1 -- 172.21.15.204:0/2019961723 learned_addr learned my addr 172.21.15.204:0/2019961723 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-03T20:28:37.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.221+0000 7f52f2a13700 1 --2- 172.21.15.204:0/2019961723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52ec103fa0 0x7f52ec1a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:28:37.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.221+0000 7f52f1a11700 1 --2- 172.21.15.204:0/2019961723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52ec103600 0x7f52ec19e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:28:37.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.221+0000 7f52f2212700 1 -- 172.21.15.204:0/2019961723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52ec103fa0 msgr2=0x7f52ec1a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:28:37.223 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.221+0000 7f52f2212700 1 --2- 172.21.15.204:0/2019961723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52ec103fa0 0x7f52ec1a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:28:37.223 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.221+0000 7f52f2212700 1 -- 172.21.15.204:0/2019961723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52ec103600 msgr2=0x7f52ec19e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:28:37.223 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.221+0000 7f52f2212700 1 --2- 172.21.15.204:0/2019961723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52ec103600 0x7f52ec19e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:28:37.223 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.221+0000 7f52f2212700 1 -- 172.21.15.204:0/2019961723 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52ec1a1290 con 0x7f52ec102400 2024-05-03T20:28:37.223 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.221+0000 7f52f1a11700 1 --2- 172.21.15.204:0/2019961723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52ec103600 0x7f52ec19e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:28:37.223 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.221+0000 7f52f2212700 1 --2- 172.21.15.204:0/2019961723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52ec102400 0x7f52ec06c780 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f52e800c970 tx=0x7f52e800cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:28:37.223 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.221+0000 7f52e37fe700 1 -- 172.21.15.204:0/2019961723 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e8004220 con 0x7f52ec102400 2024-05-03T20:28:37.223 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.221+0000 7f52e37fe700 1 -- 172.21.15.204:0/2019961723 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52e8007d50 con 0x7f52ec102400 2024-05-03T20:28:37.223 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.221+0000 7f52f4476700 1 -- 172.21.15.204:0/2019961723 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52ec1a1580 con 0x7f52ec102400 2024-05-03T20:28:37.223 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.221+0000 7f52e37fe700 1 -- 172.21.15.204:0/2019961723 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e8003d00 con 0x7f52ec102400 2024-05-03T20:28:37.223 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.221+0000 7f52f4476700 1 -- 172.21.15.204:0/2019961723 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52ec1a1b10 con 0x7f52ec102400 2024-05-03T20:28:37.224 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.222+0000 7f52e17fa700 1 -- 172.21.15.204:0/2019961723 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52d4005330 con 0x7f52ec102400 2024-05-03T20:28:37.224 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.224+0000 7f52e37fe700 1 -- 172.21.15.204:0/2019961723 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7f52e80078a0 con 0x7f52ec102400 2024-05-03T20:28:37.224 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.225+0000 7f52e37fe700 1 --2- 172.21.15.204:0/2019961723 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f52d80b2870 0x7f52d80b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:28:37.224 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.225+0000 7f52e37fe700 1 -- 172.21.15.204:0/2019961723 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7f52e80e0940 con 0x7f52ec102400 2024-05-03T20:28:37.224 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.225+0000 7f52f1a11700 1 --2- 172.21.15.204:0/2019961723 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f52d80b2870 0x7f52d80b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:28:37.225 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.225+0000 7f52f1a11700 1 --2- 172.21.15.204:0/2019961723 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f52d80b2870 0x7f52d80b4d30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f52dc006010 tx=0x7f52dc009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:28:37.228 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.229+0000 7f52e37fe700 1 -- 172.21.15.204:0/2019961723 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f52e8014030 con 0x7f52ec102400 2024-05-03T20:28:37.439 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:37.439+0000 7f52e17fa700 1 -- 172.21.15.204:0/2019961723 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f52d4000bc0 con 0x7f52d80b2870 2024-05-03T20:28:37.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:37 smithi181 ceph-mon[29801]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:37.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:37 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T20:28:37.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:37 smithi181 ceph-mon[29801]: osd.1 [v2:172.21.15.196:6800/2136873375,v1:172.21.15.196:6801/2136873375] boot 2024-05-03T20:28:37.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:37 smithi181 ceph-mon[29801]: osdmap e13: 2 total, 2 up, 2 in 2024-05-03T20:28:37.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:37 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T20:28:37.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:37 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:28:37.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:37 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:28:37.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:37 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:38.033 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:37 smithi204 ceph-mon[33595]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:38.033 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:37 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T20:28:38.033 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:37 smithi204 ceph-mon[33595]: osd.1 [v2:172.21.15.196:6800/2136873375,v1:172.21.15.196:6801/2136873375] boot 2024-05-03T20:28:38.033 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:37 smithi204 ceph-mon[33595]: osdmap e13: 2 total, 2 up, 2 in 2024-05-03T20:28:38.033 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:37 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T20:28:38.033 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:37 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:28:38.033 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:37 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:28:38.033 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:37 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:38.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:37 smithi196 ceph-mon[32858]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-03T20:28:38.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:37 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T20:28:38.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:37 smithi196 ceph-mon[32858]: osd.1 [v2:172.21.15.196:6800/2136873375,v1:172.21.15.196:6801/2136873375] boot 2024-05-03T20:28:38.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:37 smithi196 ceph-mon[32858]: osdmap e13: 2 total, 2 up, 2 in 2024-05-03T20:28:38.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:37 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T20:28:38.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:37 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:28:38.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:37 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:28:38.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:37 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:39.038 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:38 smithi181 ceph-mon[29801]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:28:39.038 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:38 smithi181 ceph-mon[29801]: osdmap e14: 2 total, 2 up, 2 in 2024-05-03T20:28:39.102 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:38 smithi204 ceph-mon[33595]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:28:39.102 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:38 smithi204 ceph-mon[33595]: osdmap e14: 2 total, 2 up, 2 in 2024-05-03T20:28:39.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:38 smithi196 ceph-mon[32858]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:28:39.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:38 smithi196 ceph-mon[32858]: osdmap e14: 2 total, 2 up, 2 in 2024-05-03T20:28:40.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:39 smithi196 ceph-mon[32858]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:40.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:39 smithi204 ceph-mon[33595]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:40.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:39 smithi181 ceph-mon[29801]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:41.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:40 smithi196 ceph-mon[32858]: from='client.? 172.21.15.204:0/1407681020' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d5573cc-37c1-4c3b-a61e-2363ef3ff964"}]: dispatch 2024-05-03T20:28:41.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:40 smithi196 ceph-mon[32858]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d5573cc-37c1-4c3b-a61e-2363ef3ff964"}]: dispatch 2024-05-03T20:28:41.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:40 smithi196 ceph-mon[32858]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d5573cc-37c1-4c3b-a61e-2363ef3ff964"}]': finished 2024-05-03T20:28:41.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:40 smithi196 ceph-mon[32858]: osdmap e15: 3 total, 2 up, 3 in 2024-05-03T20:28:41.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:40 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T20:28:41.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:40 smithi196 ceph-mon[32858]: from='client.? 172.21.15.204:0/3207790420' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:28:41.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:40 smithi204 ceph-mon[33595]: from='client.? 172.21.15.204:0/1407681020' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d5573cc-37c1-4c3b-a61e-2363ef3ff964"}]: dispatch 2024-05-03T20:28:41.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:40 smithi204 ceph-mon[33595]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d5573cc-37c1-4c3b-a61e-2363ef3ff964"}]: dispatch 2024-05-03T20:28:41.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:40 smithi204 ceph-mon[33595]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d5573cc-37c1-4c3b-a61e-2363ef3ff964"}]': finished 2024-05-03T20:28:41.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:40 smithi204 ceph-mon[33595]: osdmap e15: 3 total, 2 up, 3 in 2024-05-03T20:28:41.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:40 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T20:28:41.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:40 smithi204 ceph-mon[33595]: from='client.? 172.21.15.204:0/3207790420' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:28:41.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:40 smithi181 ceph-mon[29801]: from='client.? 172.21.15.204:0/1407681020' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d5573cc-37c1-4c3b-a61e-2363ef3ff964"}]: dispatch 2024-05-03T20:28:41.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:40 smithi181 ceph-mon[29801]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d5573cc-37c1-4c3b-a61e-2363ef3ff964"}]: dispatch 2024-05-03T20:28:41.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:40 smithi181 ceph-mon[29801]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d5573cc-37c1-4c3b-a61e-2363ef3ff964"}]': finished 2024-05-03T20:28:41.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:40 smithi181 ceph-mon[29801]: osdmap e15: 3 total, 2 up, 3 in 2024-05-03T20:28:41.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:40 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T20:28:41.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:40 smithi181 ceph-mon[29801]: from='client.? 172.21.15.204:0/3207790420' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:28:42.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:41 smithi196 ceph-mon[32858]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:42.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:41 smithi204 ceph-mon[33595]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:42.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:41 smithi181 ceph-mon[29801]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:44.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:43 smithi196 ceph-mon[32858]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:44.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:43 smithi204 ceph-mon[33595]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:44.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:43 smithi181 ceph-mon[29801]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:45.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:45 smithi204 ceph-mon[33595]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:46.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:45 smithi196 ceph-mon[32858]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:46.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:45 smithi181 ceph-mon[29801]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:48.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:47 smithi204 ceph-mon[33595]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:48.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:47 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-03T20:28:48.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:47 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:48.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:47 smithi196 ceph-mon[32858]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:48.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:47 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-03T20:28:48.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:47 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:48.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:47 smithi181 ceph-mon[29801]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:48.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:47 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-03T20:28:48.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:47 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:49.055 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:48 smithi204 ceph-mon[33595]: Deploying daemon osd.2 on smithi204 2024-05-03T20:28:49.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:48 smithi196 ceph-mon[32858]: Deploying daemon osd.2 on smithi204 2024-05-03T20:28:49.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:48 smithi181 ceph-mon[29801]: Deploying daemon osd.2 on smithi204 2024-05-03T20:28:49.858 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:49 smithi204 ceph-mon[33595]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:50.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:49 smithi196 ceph-mon[32858]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:50.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:49 smithi181 ceph-mon[29801]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:52.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:51 smithi196 ceph-mon[32858]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:52.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:51 smithi204 ceph-mon[33595]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:52.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:51 smithi181 ceph-mon[29801]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:53.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:53 smithi181 ceph-mon[29801]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:53.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:53 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:53.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:53 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:53.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:53 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:28:53.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:53 smithi196 ceph-mon[32858]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:53.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:53 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:53.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:53 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:53.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:53 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:28:53.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:53 smithi204 ceph-mon[33595]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:53.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:53 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:53.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:53 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:53.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:53 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:28:55.686 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:55 smithi204 ceph-mon[33595]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:55.686 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:55 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:55.686 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:55 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:55.686 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:55 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:55.687 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:55 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:28:55.687 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:55 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:55.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:55 smithi181 ceph-mon[29801]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:55.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:55 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:55.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:55 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:55.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:55 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:55.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:55 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:28:55.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:55 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:55.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:55 smithi196 ceph-mon[32858]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:55.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:55 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:55.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:55 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:55.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:55 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:28:55.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:55 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:28:55.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:55 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:56.054 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:56.054+0000 7f52e37fe700 1 -- 172.21.15.204:0/2019961723 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f52d4000bc0 con 0x7f52d80b2870 2024-05-03T20:28:56.054 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 2 on host 'smithi204' 2024-05-03T20:28:56.056 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:56.057+0000 7f52e17fa700 1 -- 172.21.15.204:0/2019961723 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f52d80b2870 msgr2=0x7f52d80b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:28:56.056 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:56.057+0000 7f52e17fa700 1 --2- 172.21.15.204:0/2019961723 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f52d80b2870 0x7f52d80b4d30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f52dc006010 tx=0x7f52dc009510 comp rx=0 tx=0).stop 2024-05-03T20:28:56.056 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:56.057+0000 7f52e17fa700 1 -- 172.21.15.204:0/2019961723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52ec102400 msgr2=0x7f52ec06c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:28:56.056 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:56.057+0000 7f52e17fa700 1 --2- 172.21.15.204:0/2019961723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52ec102400 0x7f52ec06c780 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f52e800c970 tx=0x7f52e800cc80 comp rx=0 tx=0).stop 2024-05-03T20:28:56.057 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:56.057+0000 7f52e17fa700 1 -- 172.21.15.204:0/2019961723 shutdown_connections 2024-05-03T20:28:56.057 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:56.057+0000 7f52e17fa700 1 --2- 172.21.15.204:0/2019961723 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f52d80b2870 0x7f52d80b4d30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:28:56.057 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:56.057+0000 7f52e17fa700 1 --2- 172.21.15.204:0/2019961723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52ec103fa0 0x7f52ec1a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:28:56.057 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:56.057+0000 7f52e17fa700 1 --2- 172.21.15.204:0/2019961723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52ec102400 0x7f52ec06c780 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:28:56.057 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:56.057+0000 7f52e17fa700 1 --2- 172.21.15.204:0/2019961723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52ec103600 0x7f52ec19e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:28:56.057 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:56.057+0000 7f52e17fa700 1 -- 172.21.15.204:0/2019961723 >> 172.21.15.204:0/2019961723 conn(0x7f52ec0fd9c0 msgr2=0x7f52ec106a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:28:56.057 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:56.057+0000 7f52e17fa700 1 -- 172.21.15.204:0/2019961723 shutdown_connections 2024-05-03T20:28:56.057 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:28:56.057+0000 7f52e17fa700 1 -- 172.21.15.204:0/2019961723 wait complete. 2024-05-03T20:28:56.648 DEBUG:teuthology.orchestra.run.smithi204:osd.2> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.2.service 2024-05-03T20:28:56.651 INFO:tasks.cephadm:Deploying osd.3 on smithi181 with /dev/vg_nvme/lv_3... 2024-05-03T20:28:56.651 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-03T20:28:56.898 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:28:57.324 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:57 smithi181 ceph-mon[29801]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:57.324 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:57 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:57.324 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:57 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:57.324 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:57 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:28:57.324 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:57 smithi181 ceph-mon[29801]: from='osd.2 [v2:172.21.15.204:6800/3291673137,v1:172.21.15.204:6801/3291673137]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T20:28:57.324 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:57 smithi181 ceph-mon[29801]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T20:28:57.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:57 smithi196 ceph-mon[32858]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:57.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:57 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:57.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:57 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:57.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:57 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:28:57.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:57 smithi196 ceph-mon[32858]: from='osd.2 [v2:172.21.15.204:6800/3291673137,v1:172.21.15.204:6801/3291673137]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T20:28:57.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:57 smithi196 ceph-mon[32858]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T20:28:57.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:57 smithi204 ceph-mon[33595]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:57.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:57 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:57.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:57 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:28:57.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:57 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:28:57.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:57 smithi204 ceph-mon[33595]: from='osd.2 [v2:172.21.15.204:6800/3291673137,v1:172.21.15.204:6801/3291673137]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T20:28:57.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:57 smithi204 ceph-mon[33595]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T20:28:58.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:58 smithi204 ceph-mon[33595]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-03T20:28:58.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:58 smithi204 ceph-mon[33595]: osdmap e16: 3 total, 2 up, 3 in 2024-05-03T20:28:58.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:58 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T20:28:58.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:58 smithi204 ceph-mon[33595]: from='osd.2 [v2:172.21.15.204:6800/3291673137,v1:172.21.15.204:6801/3291673137]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-03T20:28:58.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:58 smithi204 ceph-mon[33595]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-03T20:28:58.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:58 smithi196 ceph-mon[32858]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-03T20:28:58.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:58 smithi196 ceph-mon[32858]: osdmap e16: 3 total, 2 up, 3 in 2024-05-03T20:28:58.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:58 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T20:28:58.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:58 smithi196 ceph-mon[32858]: from='osd.2 [v2:172.21.15.204:6800/3291673137,v1:172.21.15.204:6801/3291673137]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-03T20:28:58.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:58 smithi196 ceph-mon[32858]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-03T20:28:58.384 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:58 smithi181 ceph-mon[29801]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-03T20:28:58.384 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:58 smithi181 ceph-mon[29801]: osdmap e16: 3 total, 2 up, 3 in 2024-05-03T20:28:58.384 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:58 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T20:28:58.384 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:58 smithi181 ceph-mon[29801]: from='osd.2 [v2:172.21.15.204:6800/3291673137,v1:172.21.15.204:6801/3291673137]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-03T20:28:58.384 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:58 smithi181 ceph-mon[29801]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-03T20:28:58.961 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-03T20:28:58.961 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-03T20:28:58.961 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2024-05-03T20:28:58.961 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2024-05-03T20:28:58.961 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114648 s, 915 MB/s 2024-05-03T20:28:58.961 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2024-05-03T20:28:59.283 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:59 smithi181 ceph-mon[29801]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:59.283 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:59 smithi181 ceph-mon[29801]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-03T20:28:59.283 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:59 smithi181 ceph-mon[29801]: osdmap e17: 3 total, 2 up, 3 in 2024-05-03T20:28:59.283 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:59 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T20:28:59.283 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:59 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T20:28:59.284 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:28:59 smithi181 ceph-mon[29801]: from='osd.2 ' entity='osd.2' 2024-05-03T20:28:59.286 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:59 smithi204 ceph-mon[33595]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:59.286 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:59 smithi204 ceph-mon[33595]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-03T20:28:59.286 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:59 smithi204 ceph-mon[33595]: osdmap e17: 3 total, 2 up, 3 in 2024-05-03T20:28:59.286 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:59 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T20:28:59.286 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:59 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T20:28:59.286 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:28:59 smithi204 ceph-mon[33595]: from='osd.2 ' entity='osd.2' 2024-05-03T20:28:59.286 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 03 20:28:58 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-2[38742]: 2024-05-03T20:28:58.981+0000 7efc2698a700 -1 osd.2 0 waiting for initial osdmap 2024-05-03T20:28:59.287 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 03 20:28:59 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-2[38742]: 2024-05-03T20:28:59.123+0000 7efc1df74700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T20:28:59.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:59 smithi196 ceph-mon[32858]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:28:59.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:59 smithi196 ceph-mon[32858]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-03T20:28:59.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:59 smithi196 ceph-mon[32858]: osdmap e17: 3 total, 2 up, 3 in 2024-05-03T20:28:59.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:59 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T20:28:59.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:59 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T20:28:59.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:28:59 smithi196 ceph-mon[32858]: from='osd.2 ' entity='osd.2' 2024-05-03T20:28:59.605 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_3 2024-05-03T20:28:59.864 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:29:00.195 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:00 smithi181 ceph-mon[29801]: purged_snaps scrub starts 2024-05-03T20:29:00.196 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:00 smithi181 ceph-mon[29801]: purged_snaps scrub ok 2024-05-03T20:29:00.196 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:00 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T20:29:00.196 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:00 smithi181 ceph-mon[29801]: Detected new or changed devices on smithi204 2024-05-03T20:29:00.196 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:00 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:00.196 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:00 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:00.196 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:00 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:29:00.196 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:00 smithi181 ceph-mon[29801]: Adjusting osd_memory_target on smithi204 to 17088M 2024-05-03T20:29:00.196 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:00 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:00.196 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:00 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:00.196 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:00 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:29:00.196 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:00 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:00.196 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:00 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T20:29:00.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:00 smithi196 ceph-mon[32858]: purged_snaps scrub starts 2024-05-03T20:29:00.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:00 smithi196 ceph-mon[32858]: purged_snaps scrub ok 2024-05-03T20:29:00.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:00 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T20:29:00.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:00 smithi196 ceph-mon[32858]: Detected new or changed devices on smithi204 2024-05-03T20:29:00.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:00 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:00.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:00 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:00.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:00 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:29:00.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:00 smithi196 ceph-mon[32858]: Adjusting osd_memory_target on smithi204 to 17088M 2024-05-03T20:29:00.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:00 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:00.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:00 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:00.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:00 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:29:00.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:00 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:00.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:00 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T20:29:00.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:00 smithi204 ceph-mon[33595]: purged_snaps scrub starts 2024-05-03T20:29:00.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:00 smithi204 ceph-mon[33595]: purged_snaps scrub ok 2024-05-03T20:29:00.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:00 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T20:29:00.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:00 smithi204 ceph-mon[33595]: Detected new or changed devices on smithi204 2024-05-03T20:29:00.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:00 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:00.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:00 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:00.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:00 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:29:00.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:00 smithi204 ceph-mon[33595]: Adjusting osd_memory_target on smithi204 to 17088M 2024-05-03T20:29:00.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:00 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:00.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:00 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:00.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:00 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:29:00.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:00 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:00.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:00 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T20:29:01.162 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:01 smithi181 ceph-mon[29801]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:29:01.162 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:01 smithi181 ceph-mon[29801]: osd.2 [v2:172.21.15.204:6800/3291673137,v1:172.21.15.204:6801/3291673137] boot 2024-05-03T20:29:01.162 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:01 smithi181 ceph-mon[29801]: osdmap e18: 3 total, 3 up, 3 in 2024-05-03T20:29:01.162 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:01 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T20:29:01.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.355+0000 7f5d3eeea700 1 -- 172.21.15.181:0/1626171188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d38104850 msgr2=0x7f5d38104cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:29:01.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.355+0000 7f5d3eeea700 1 --2- 172.21.15.181:0/1626171188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d38104850 0x7f5d38104cb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5d240099c0 tx=0x7f5d24009cd0 comp rx=0 tx=0).stop 2024-05-03T20:29:01.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.355+0000 7f5d3eeea700 1 -- 172.21.15.181:0/1626171188 shutdown_connections 2024-05-03T20:29:01.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.355+0000 7f5d3eeea700 1 --2- 172.21.15.181:0/1626171188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d381051f0 0x7f5d3810baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:29:01.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.355+0000 7f5d3eeea700 1 --2- 172.21.15.181:0/1626171188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d381017d0 0x7f5d38101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:29:01.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.355+0000 7f5d3eeea700 1 --2- 172.21.15.181:0/1626171188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d38104850 0x7f5d38104cb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:29:01.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.355+0000 7f5d3eeea700 1 -- 172.21.15.181:0/1626171188 >> 172.21.15.181:0/1626171188 conn(0x7f5d380fd130 msgr2=0x7f5d380ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:29:01.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.355+0000 7f5d3eeea700 1 -- 172.21.15.181:0/1626171188 shutdown_connections 2024-05-03T20:29:01.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.355+0000 7f5d3eeea700 1 -- 172.21.15.181:0/1626171188 wait complete. 2024-05-03T20:29:01.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.356+0000 7f5d3eeea700 1 Processor -- start 2024-05-03T20:29:01.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.356+0000 7f5d3eeea700 1 -- start start 2024-05-03T20:29:01.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.356+0000 7f5d3eeea700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d381017d0 0x7f5d381041b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:29:01.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.356+0000 7f5d3eeea700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d38104850 0x7f5d38102800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:29:01.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.356+0000 7f5d3eeea700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d381051f0 0x7f5d38102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:29:01.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.356+0000 7f5d3eeea700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3810f080 con 0x7f5d381051f0 2024-05-03T20:29:01.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.356+0000 7f5d3eeea700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3810ef00 con 0x7f5d381017d0 2024-05-03T20:29:01.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.356+0000 7f5d3eeea700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3810f200 con 0x7f5d38104850 2024-05-03T20:29:01.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.356+0000 7f5d3d487700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d381051f0 0x7f5d38102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:29:01.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.356+0000 7f5d3d487700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d381051f0 0x7f5d38102d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43018/0 (socket says 172.21.15.181:43018) 2024-05-03T20:29:01.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.356+0000 7f5d3d487700 1 -- 172.21.15.181:0/4025555690 learned_addr learned my addr 172.21.15.181:0/4025555690 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:29:01.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.356+0000 7f5d37fff700 1 --2- 172.21.15.181:0/4025555690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d38104850 0x7f5d38102800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:29:01.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.356+0000 7f5d3cc86700 1 --2- 172.21.15.181:0/4025555690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d381017d0 0x7f5d381041b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:29:01.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.356+0000 7f5d3d487700 1 -- 172.21.15.181:0/4025555690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d38104850 msgr2=0x7f5d38102800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:29:01.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.356+0000 7f5d3d487700 1 --2- 172.21.15.181:0/4025555690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d38104850 0x7f5d38102800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:29:01.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.356+0000 7f5d3d487700 1 -- 172.21.15.181:0/4025555690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d381017d0 msgr2=0x7f5d381041b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:29:01.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.356+0000 7f5d3d487700 1 --2- 172.21.15.181:0/4025555690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d381017d0 0x7f5d381041b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:29:01.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.356+0000 7f5d3d487700 1 -- 172.21.15.181:0/4025555690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d381035f0 con 0x7f5d381051f0 2024-05-03T20:29:01.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.357+0000 7f5d3d487700 1 --2- 172.21.15.181:0/4025555690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d381051f0 0x7f5d38102d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000b970 tx=0x7f5d3000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:29:01.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.357+0000 7f5d35ffb700 1 -- 172.21.15.181:0/4025555690 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3000d860 con 0x7f5d381051f0 2024-05-03T20:29:01.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.357+0000 7f5d35ffb700 1 -- 172.21.15.181:0/4025555690 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d3000e430 con 0x7f5d381051f0 2024-05-03T20:29:01.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.357+0000 7f5d3eeea700 1 -- 172.21.15.181:0/4025555690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d381a3850 con 0x7f5d381051f0 2024-05-03T20:29:01.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.357+0000 7f5d35ffb700 1 -- 172.21.15.181:0/4025555690 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30017660 con 0x7f5d381051f0 2024-05-03T20:29:01.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.357+0000 7f5d3eeea700 1 -- 172.21.15.181:0/4025555690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d381a3de0 con 0x7f5d381051f0 2024-05-03T20:29:01.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.358+0000 7f5d3eeea700 1 -- 172.21.15.181:0/4025555690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d3806a190 con 0x7f5d381051f0 2024-05-03T20:29:01.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.359+0000 7f5d35ffb700 1 -- 172.21.15.181:0/4025555690 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7f5d3000da00 con 0x7f5d381051f0 2024-05-03T20:29:01.362 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:01 smithi196 ceph-mon[32858]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:29:01.362 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:01 smithi196 ceph-mon[32858]: osd.2 [v2:172.21.15.204:6800/3291673137,v1:172.21.15.204:6801/3291673137] boot 2024-05-03T20:29:01.362 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:01 smithi196 ceph-mon[32858]: osdmap e18: 3 total, 3 up, 3 in 2024-05-03T20:29:01.362 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:01 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T20:29:01.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.360+0000 7f5d35ffb700 1 --2- 172.21.15.181:0/4025555690 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5d280b2870 0x7f5d280b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:29:01.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.360+0000 7f5d3cc86700 1 --2- 172.21.15.181:0/4025555690 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5d280b2870 0x7f5d280b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:29:01.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.360+0000 7f5d35ffb700 1 -- 172.21.15.181:0/4025555690 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 2851+0+0 (secure 0 0 0) 0x7f5d300e0790 con 0x7f5d381051f0 2024-05-03T20:29:01.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.360+0000 7f5d3cc86700 1 --2- 172.21.15.181:0/4025555690 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5d280b2870 0x7f5d280b4d30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5d3806aec0 tx=0x7f5d2c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:29:01.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:01 smithi204 ceph-mon[33595]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T20:29:01.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:01 smithi204 ceph-mon[33595]: osd.2 [v2:172.21.15.204:6800/3291673137,v1:172.21.15.204:6801/3291673137] boot 2024-05-03T20:29:01.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:01 smithi204 ceph-mon[33595]: osdmap e18: 3 total, 3 up, 3 in 2024-05-03T20:29:01.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:01 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T20:29:01.364 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.364+0000 7f5d35ffb700 1 -- 172.21.15.181:0/4025555690 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5d30013030 con 0x7f5d381051f0 2024-05-03T20:29:01.576 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:01.577+0000 7f5d3eeea700 1 -- 172.21.15.181:0/4025555690 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f5d380638e0 con 0x7f5d280b2870 2024-05-03T20:29:02.334 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:02 smithi181 ceph-mon[29801]: osdmap e19: 3 total, 3 up, 3 in 2024-05-03T20:29:02.334 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:02 smithi181 ceph-mon[29801]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:29:02.334 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:02 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:29:02.334 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:02 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:29:02.334 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:02 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:02.335 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:02 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-03T20:29:02.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:02 smithi196 ceph-mon[32858]: osdmap e19: 3 total, 3 up, 3 in 2024-05-03T20:29:02.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:02 smithi196 ceph-mon[32858]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:29:02.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:02 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:29:02.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:02 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:29:02.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:02 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:02.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:02 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-03T20:29:02.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:02 smithi204 ceph-mon[33595]: osdmap e19: 3 total, 3 up, 3 in 2024-05-03T20:29:02.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:02 smithi204 ceph-mon[33595]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:29:02.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:02 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:29:02.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:02 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:29:02.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:02 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:02.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:02 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-03T20:29:03.140 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:03 smithi181 ceph-mon[29801]: pgmap v83: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:03.141 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:03 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-03T20:29:03.141 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:03 smithi181 ceph-mon[29801]: osdmap e20: 3 total, 3 up, 3 in 2024-05-03T20:29:03.141 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:03 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T20:29:03.141 INFO:journalctl@ceph.osd.0.smithi181.stdout:May 03 20:29:03 smithi181 sudo[49761]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T20:29:03.141 INFO:journalctl@ceph.osd.0.smithi181.stdout:May 03 20:29:03 smithi181 sudo[49761]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T20:29:03.141 INFO:journalctl@ceph.osd.0.smithi181.stdout:May 03 20:29:03 smithi181 sudo[49761]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T20:29:03.141 INFO:journalctl@ceph.osd.0.smithi181.stdout:May 03 20:29:03 smithi181 sudo[49761]: pam_unix(sudo:session): session closed for user root 2024-05-03T20:29:03.141 INFO:journalctl@ceph.osd.0.smithi181.stdout:May 03 20:29:03 smithi181 sudo[49764]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T20:29:03.141 INFO:journalctl@ceph.osd.0.smithi181.stdout:May 03 20:29:03 smithi181 sudo[49764]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T20:29:03.141 INFO:journalctl@ceph.osd.0.smithi181.stdout:May 03 20:29:03 smithi181 sudo[49764]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T20:29:03.351 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:03 smithi204 ceph-mon[33595]: pgmap v83: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:03.352 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:03 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-03T20:29:03.352 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:03 smithi204 ceph-mon[33595]: osdmap e20: 3 total, 3 up, 3 in 2024-05-03T20:29:03.352 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:03 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T20:29:03.352 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 03 20:29:03 smithi204 sudo[42494]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T20:29:03.352 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 03 20:29:03 smithi204 sudo[42494]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T20:29:03.352 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 03 20:29:03 smithi204 sudo[42494]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T20:29:03.352 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 03 20:29:03 smithi204 sudo[42494]: pam_unix(sudo:session): session closed for user root 2024-05-03T20:29:03.352 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 03 20:29:03 smithi204 sudo[42497]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T20:29:03.352 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 03 20:29:03 smithi204 sudo[42497]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T20:29:03.353 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 03 20:29:03 smithi204 sudo[42497]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T20:29:03.353 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 03 20:29:03 smithi204 sudo[42497]: pam_unix(sudo:session): session closed for user root 2024-05-03T20:29:03.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:03 smithi196 ceph-mon[32858]: pgmap v83: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:03.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:03 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-03T20:29:03.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:03 smithi196 ceph-mon[32858]: osdmap e20: 3 total, 3 up, 3 in 2024-05-03T20:29:03.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:03 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T20:29:03.357 INFO:journalctl@ceph.osd.1.smithi196.stdout:May 03 20:29:03 smithi196 sudo[41616]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T20:29:03.357 INFO:journalctl@ceph.osd.1.smithi196.stdout:May 03 20:29:03 smithi196 sudo[41616]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T20:29:03.357 INFO:journalctl@ceph.osd.1.smithi196.stdout:May 03 20:29:03 smithi196 sudo[41616]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T20:29:03.357 INFO:journalctl@ceph.osd.1.smithi196.stdout:May 03 20:29:03 smithi196 sudo[41616]: pam_unix(sudo:session): session closed for user root 2024-05-03T20:29:03.357 INFO:journalctl@ceph.osd.1.smithi196.stdout:May 03 20:29:03 smithi196 sudo[41619]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T20:29:03.357 INFO:journalctl@ceph.osd.1.smithi196.stdout:May 03 20:29:03 smithi196 sudo[41619]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T20:29:03.357 INFO:journalctl@ceph.osd.1.smithi196.stdout:May 03 20:29:03 smithi196 sudo[41619]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T20:29:03.358 INFO:journalctl@ceph.osd.1.smithi196.stdout:May 03 20:29:03 smithi196 sudo[41619]: pam_unix(sudo:session): session closed for user root 2024-05-03T20:29:03.446 INFO:journalctl@ceph.osd.0.smithi181.stdout:May 03 20:29:03 smithi181 sudo[49764]: pam_unix(sudo:session): session closed for user root 2024-05-03T20:29:03.446 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:03 smithi181 sudo[49788]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T20:29:03.446 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:03 smithi181 sudo[49788]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T20:29:03.446 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:03 smithi181 sudo[49788]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T20:29:03.446 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:03 smithi181 sudo[49788]: pam_unix(sudo:session): session closed for user root 2024-05-03T20:29:03.447 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:03 smithi181 sudo[49791]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T20:29:03.447 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:03 smithi181 sudo[49791]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T20:29:03.447 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:03 smithi181 sudo[49791]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T20:29:03.447 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:03 smithi181 sudo[49791]: pam_unix(sudo:session): session closed for user root 2024-05-03T20:29:03.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:03 smithi204 sudo[42500]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T20:29:03.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:03 smithi204 sudo[42500]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T20:29:03.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:03 smithi204 sudo[42500]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T20:29:03.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:03 smithi204 sudo[42500]: pam_unix(sudo:session): session closed for user root 2024-05-03T20:29:03.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:03 smithi204 sudo[42503]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T20:29:03.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:03 smithi204 sudo[42503]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T20:29:03.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:03 smithi204 sudo[42503]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T20:29:03.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:03 smithi204 sudo[42503]: pam_unix(sudo:session): session closed for user root 2024-05-03T20:29:03.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:03 smithi196 sudo[41622]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T20:29:03.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:03 smithi196 sudo[41622]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T20:29:03.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:03 smithi196 sudo[41622]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T20:29:03.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:03 smithi196 sudo[41622]: pam_unix(sudo:session): session closed for user root 2024-05-03T20:29:03.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:03 smithi196 sudo[41625]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T20:29:03.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:03 smithi196 sudo[41625]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T20:29:03.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:03 smithi196 sudo[41625]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T20:29:03.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:03 smithi196 sudo[41625]: pam_unix(sudo:session): session closed for user root 2024-05-03T20:29:04.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:04.321+0000 7f5d35ffb700 1 -- 172.21.15.181:0/4025555690 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f5d300a5670 con 0x7f5d381051f0 2024-05-03T20:29:04.322 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:04 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-03T20:29:04.322 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:04 smithi181 ceph-mon[29801]: osdmap e21: 3 total, 3 up, 3 in 2024-05-03T20:29:04.322 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:04 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:04.322 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:04 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:04.322 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:04 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:04.322 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:04 smithi181 ceph-mon[29801]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T20:29:04.322 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:04 smithi181 ceph-mon[29801]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T20:29:04.322 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:04 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T20:29:04.322 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:04 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:29:04.322 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:04 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T20:29:04.323 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:04 smithi181 ceph-mon[29801]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T20:29:04.323 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:04 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T20:29:04.323 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:04 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:29:04.323 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:04 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T20:29:04.323 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:04 smithi181 ceph-mon[29801]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T20:29:04.323 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:04 smithi181 ceph-mon[29801]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T20:29:04.323 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:04 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T20:29:04.323 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:04 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:29:04.323 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:04 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T20:29:04.323 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:04 smithi181 ceph-mon[29801]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T20:29:04.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:04 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-03T20:29:04.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:04 smithi196 ceph-mon[32858]: osdmap e21: 3 total, 3 up, 3 in 2024-05-03T20:29:04.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:04 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:04.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:04 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:04.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:04 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:04.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:04 smithi196 ceph-mon[32858]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T20:29:04.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:04 smithi196 ceph-mon[32858]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T20:29:04.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:04 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T20:29:04.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:04 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:29:04.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:04 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T20:29:04.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:04 smithi196 ceph-mon[32858]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T20:29:04.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:04 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T20:29:04.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:04 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:29:04.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:04 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T20:29:04.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:04 smithi196 ceph-mon[32858]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T20:29:04.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:04 smithi196 ceph-mon[32858]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T20:29:04.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:04 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T20:29:04.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:04 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:29:04.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:04 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T20:29:04.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:04 smithi196 ceph-mon[32858]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T20:29:04.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:04 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-03T20:29:04.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:04 smithi204 ceph-mon[33595]: osdmap e21: 3 total, 3 up, 3 in 2024-05-03T20:29:04.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:04 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:04.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:04 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:04.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:04 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:04.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:04 smithi204 ceph-mon[33595]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T20:29:04.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:04 smithi204 ceph-mon[33595]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T20:29:04.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:04 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T20:29:04.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:04 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:29:04.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:04 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T20:29:04.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:04 smithi204 ceph-mon[33595]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T20:29:04.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:04 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T20:29:04.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:04 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:29:04.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:04 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T20:29:04.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:04 smithi204 ceph-mon[33595]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T20:29:04.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:04 smithi204 ceph-mon[33595]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T20:29:04.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:04 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T20:29:04.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:04 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T20:29:04.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:04 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T20:29:04.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:04 smithi204 ceph-mon[33595]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T20:29:05.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:05 smithi196 ceph-mon[32858]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:05.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:05 smithi196 ceph-mon[32858]: osdmap e22: 3 total, 3 up, 3 in 2024-05-03T20:29:05.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:05 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/578231631' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9b205e3-a335-4771-9a66-55cfe6a8c31a"}]: dispatch 2024-05-03T20:29:05.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:05 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/578231631' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9b205e3-a335-4771-9a66-55cfe6a8c31a"}]': finished 2024-05-03T20:29:05.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:05 smithi196 ceph-mon[32858]: osdmap e23: 4 total, 3 up, 4 in 2024-05-03T20:29:05.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:05 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T20:29:05.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:05 smithi196 ceph-mon[32858]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-03T20:29:05.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:05 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3684786581' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:29:05.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:05 smithi204 ceph-mon[33595]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:05.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:05 smithi204 ceph-mon[33595]: osdmap e22: 3 total, 3 up, 3 in 2024-05-03T20:29:05.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:05 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/578231631' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9b205e3-a335-4771-9a66-55cfe6a8c31a"}]: dispatch 2024-05-03T20:29:05.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:05 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/578231631' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9b205e3-a335-4771-9a66-55cfe6a8c31a"}]': finished 2024-05-03T20:29:05.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:05 smithi204 ceph-mon[33595]: osdmap e23: 4 total, 3 up, 4 in 2024-05-03T20:29:05.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:05 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T20:29:05.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:05 smithi204 ceph-mon[33595]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-03T20:29:05.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:05 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3684786581' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:29:05.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:05 smithi181 ceph-mon[29801]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:05.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:05 smithi181 ceph-mon[29801]: osdmap e22: 3 total, 3 up, 3 in 2024-05-03T20:29:05.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:05 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/578231631' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9b205e3-a335-4771-9a66-55cfe6a8c31a"}]: dispatch 2024-05-03T20:29:05.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:05 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/578231631' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9b205e3-a335-4771-9a66-55cfe6a8c31a"}]': finished 2024-05-03T20:29:05.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:05 smithi181 ceph-mon[29801]: osdmap e23: 4 total, 3 up, 4 in 2024-05-03T20:29:05.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:05 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T20:29:05.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:05 smithi181 ceph-mon[29801]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-03T20:29:05.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:05 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3684786581' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:29:07.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:07 smithi196 ceph-mon[32858]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:07.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:07 smithi204 ceph-mon[33595]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:07.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:07 smithi181 ceph-mon[29801]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:09.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:09 smithi181 ceph-mon[29801]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:09.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:09 smithi196 ceph-mon[32858]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:09.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:09 smithi204 ceph-mon[33595]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:11.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:11 smithi181 ceph-mon[29801]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:11.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:11 smithi196 ceph-mon[32858]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:11.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:11 smithi204 ceph-mon[33595]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:12.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-03T20:29:12.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:12.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:12 smithi196 ceph-mon[32858]: Deploying daemon osd.3 on smithi181 2024-05-03T20:29:12.363 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-03T20:29:12.363 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:12.363 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:12 smithi181 ceph-mon[29801]: Deploying daemon osd.3 on smithi181 2024-05-03T20:29:12.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-03T20:29:12.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:12.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:12 smithi204 ceph-mon[33595]: Deploying daemon osd.3 on smithi181 2024-05-03T20:29:13.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:13 smithi196 ceph-mon[32858]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:13.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:13 smithi204 ceph-mon[33595]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:13.428 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:13 smithi181 ceph-mon[29801]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:15.326 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:15 smithi181 ceph-mon[29801]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:15.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:15 smithi196 ceph-mon[32858]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:15.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:15 smithi204 ceph-mon[33595]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:17.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:17 smithi196 ceph-mon[32858]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:17.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:17 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:17.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:17 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:17.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:17 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:29:17.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:17 smithi204 ceph-mon[33595]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:17.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:17 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:17.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:17 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:17.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:17 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:29:17.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:17 smithi181 ceph-mon[29801]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:17.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:17 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:17.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:17 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:17.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:17 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:29:19.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:19 smithi196 ceph-mon[32858]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:19.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:19 smithi204 ceph-mon[33595]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:19.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:19 smithi181 ceph-mon[29801]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:20.126 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 3 on host 'smithi181' 2024-05-03T20:29:20.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:20.125+0000 7f5d35ffb700 1 -- 172.21.15.181:0/4025555690 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5d380638e0 con 0x7f5d280b2870 2024-05-03T20:29:20.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:20.128+0000 7f5d3eeea700 1 -- 172.21.15.181:0/4025555690 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5d280b2870 msgr2=0x7f5d280b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:29:20.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:20.128+0000 7f5d3eeea700 1 --2- 172.21.15.181:0/4025555690 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5d280b2870 0x7f5d280b4d30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5d3806aec0 tx=0x7f5d2c009190 comp rx=0 tx=0).stop 2024-05-03T20:29:20.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:20.128+0000 7f5d3eeea700 1 -- 172.21.15.181:0/4025555690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d381051f0 msgr2=0x7f5d38102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:29:20.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:20.128+0000 7f5d3eeea700 1 --2- 172.21.15.181:0/4025555690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d381051f0 0x7f5d38102d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000b970 tx=0x7f5d3000bc80 comp rx=0 tx=0).stop 2024-05-03T20:29:20.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:20.128+0000 7f5d3eeea700 1 -- 172.21.15.181:0/4025555690 shutdown_connections 2024-05-03T20:29:20.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:20.128+0000 7f5d3eeea700 1 --2- 172.21.15.181:0/4025555690 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5d280b2870 0x7f5d280b4d30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:29:20.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:20.128+0000 7f5d3eeea700 1 --2- 172.21.15.181:0/4025555690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d381051f0 0x7f5d38102d40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:29:20.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:20.128+0000 7f5d3eeea700 1 --2- 172.21.15.181:0/4025555690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d381017d0 0x7f5d381041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:29:20.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:20.128+0000 7f5d3eeea700 1 --2- 172.21.15.181:0/4025555690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d38104850 0x7f5d38102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:29:20.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:20.128+0000 7f5d3eeea700 1 -- 172.21.15.181:0/4025555690 >> 172.21.15.181:0/4025555690 conn(0x7f5d380fd130 msgr2=0x7f5d38107c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:29:20.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:20.128+0000 7f5d3eeea700 1 -- 172.21.15.181:0/4025555690 shutdown_connections 2024-05-03T20:29:20.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:29:20.128+0000 7f5d3eeea700 1 -- 172.21.15.181:0/4025555690 wait complete. 2024-05-03T20:29:20.681 DEBUG:teuthology.orchestra.run.smithi181:osd.3> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.3.service 2024-05-03T20:29:20.683 INFO:tasks.cephadm:Deploying osd.4 on smithi196 with /dev/vg_nvme/lv_3... 2024-05-03T20:29:20.683 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-03T20:29:20.930 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T20:29:21.104 INFO:journalctl@ceph.osd.3.smithi181.stdout:May 03 20:29:20 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-3[51875]: 2024-05-03T20:29:20.739+0000 7fc4ebfa77c0 -1 osd.3 0 log_to_monitors true 2024-05-03T20:29:21.342 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:21 smithi196 ceph-mon[32858]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:21.342 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:21 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:21.342 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:21 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:21.342 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:21 smithi196 ceph-mon[32858]: from='osd.3 [v2:172.21.15.181:6810/29064971,v1:172.21.15.181:6811/29064971]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-03T20:29:21.342 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:21 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:21.342 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:21 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:21.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:21 smithi204 ceph-mon[33595]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:21.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:21 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:21.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:21 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:21.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:21 smithi204 ceph-mon[33595]: from='osd.3 [v2:172.21.15.181:6810/29064971,v1:172.21.15.181:6811/29064971]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-03T20:29:21.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:21 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:21.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:21 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:21.433 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:21 smithi181 ceph-mon[29801]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:21.434 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:21 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:21.434 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:21 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:21.434 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:21 smithi181 ceph-mon[29801]: from='osd.3 [v2:172.21.15.181:6810/29064971,v1:172.21.15.181:6811/29064971]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-03T20:29:21.434 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:21 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:21.434 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:21 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:22.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:22 smithi196 ceph-mon[32858]: from='osd.3 [v2:172.21.15.181:6810/29064971,v1:172.21.15.181:6811/29064971]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-03T20:29:22.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:22 smithi196 ceph-mon[32858]: osdmap e24: 4 total, 3 up, 4 in 2024-05-03T20:29:22.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:22 smithi196 ceph-mon[32858]: from='osd.3 [v2:172.21.15.181:6810/29064971,v1:172.21.15.181:6811/29064971]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-03T20:29:22.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:22 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T20:29:22.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:22 smithi181 ceph-mon[29801]: from='osd.3 [v2:172.21.15.181:6810/29064971,v1:172.21.15.181:6811/29064971]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-03T20:29:22.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:22 smithi181 ceph-mon[29801]: osdmap e24: 4 total, 3 up, 4 in 2024-05-03T20:29:22.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:22 smithi181 ceph-mon[29801]: from='osd.3 [v2:172.21.15.181:6810/29064971,v1:172.21.15.181:6811/29064971]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-03T20:29:22.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:22 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T20:29:22.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:22 smithi204 ceph-mon[33595]: from='osd.3 [v2:172.21.15.181:6810/29064971,v1:172.21.15.181:6811/29064971]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-03T20:29:22.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:22 smithi204 ceph-mon[33595]: osdmap e24: 4 total, 3 up, 4 in 2024-05-03T20:29:22.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:22 smithi204 ceph-mon[33595]: from='osd.3 [v2:172.21.15.181:6810/29064971,v1:172.21.15.181:6811/29064971]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-03T20:29:22.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:22 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T20:29:22.915 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-03T20:29:22.915 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-03T20:29:22.915 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-05-03T20:29:22.915 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-05-03T20:29:22.915 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128927 s, 813 MB/s 2024-05-03T20:29:22.915 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-05-03T20:29:23.463 INFO:journalctl@ceph.osd.3.smithi181.stdout:May 03 20:29:23 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-3[51875]: 2024-05-03T20:29:23.014+0000 7fc4e160b700 -1 osd.3 0 waiting for initial osdmap 2024-05-03T20:29:23.463 INFO:journalctl@ceph.osd.3.smithi181.stdout:May 03 20:29:23 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-3[51875]: 2024-05-03T20:29:23.187+0000 7fc4d93f6700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T20:29:23.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:23 smithi181 ceph-mon[29801]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:23.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:23 smithi181 ceph-mon[29801]: from='osd.3 [v2:172.21.15.181:6810/29064971,v1:172.21.15.181:6811/29064971]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-03T20:29:23.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:23 smithi181 ceph-mon[29801]: osdmap e25: 4 total, 3 up, 4 in 2024-05-03T20:29:23.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:23 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T20:29:23.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:23 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T20:29:23.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:23 smithi181 ceph-mon[29801]: from='osd.3 [v2:172.21.15.181:6810/29064971,v1:172.21.15.181:6811/29064971]' entity='osd.3' 2024-05-03T20:29:23.468 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:23 smithi196 ceph-mon[32858]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:23.468 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:23 smithi196 ceph-mon[32858]: from='osd.3 [v2:172.21.15.181:6810/29064971,v1:172.21.15.181:6811/29064971]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-03T20:29:23.468 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:23 smithi196 ceph-mon[32858]: osdmap e25: 4 total, 3 up, 4 in 2024-05-03T20:29:23.468 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:23 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T20:29:23.468 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:23 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T20:29:23.468 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:23 smithi196 ceph-mon[32858]: from='osd.3 [v2:172.21.15.181:6810/29064971,v1:172.21.15.181:6811/29064971]' entity='osd.3' 2024-05-03T20:29:23.495 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_3 2024-05-03T20:29:23.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:23 smithi204 ceph-mon[33595]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:23.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:23 smithi204 ceph-mon[33595]: from='osd.3 [v2:172.21.15.181:6810/29064971,v1:172.21.15.181:6811/29064971]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-03T20:29:23.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:23 smithi204 ceph-mon[33595]: osdmap e25: 4 total, 3 up, 4 in 2024-05-03T20:29:23.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:23 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T20:29:23.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:23 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T20:29:23.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:23 smithi204 ceph-mon[33595]: from='osd.3 [v2:172.21.15.181:6810/29064971,v1:172.21.15.181:6811/29064971]' entity='osd.3' 2024-05-03T20:29:23.740 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T20:29:24.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:24 smithi196 ceph-mon[32858]: purged_snaps scrub starts 2024-05-03T20:29:24.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:24 smithi196 ceph-mon[32858]: purged_snaps scrub ok 2024-05-03T20:29:24.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:24 smithi196 ceph-mon[32858]: osdmap e26: 4 total, 3 up, 4 in 2024-05-03T20:29:24.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:24 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T20:29:24.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:24 smithi196 ceph-mon[32858]: osd.3 [v2:172.21.15.181:6810/29064971,v1:172.21.15.181:6811/29064971] boot 2024-05-03T20:29:24.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:24 smithi196 ceph-mon[32858]: osdmap e27: 4 total, 4 up, 4 in 2024-05-03T20:29:24.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:24 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T20:29:24.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:24 smithi204 ceph-mon[33595]: purged_snaps scrub starts 2024-05-03T20:29:24.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:24 smithi204 ceph-mon[33595]: purged_snaps scrub ok 2024-05-03T20:29:24.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:24 smithi204 ceph-mon[33595]: osdmap e26: 4 total, 3 up, 4 in 2024-05-03T20:29:24.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:24 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T20:29:24.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:24 smithi204 ceph-mon[33595]: osd.3 [v2:172.21.15.181:6810/29064971,v1:172.21.15.181:6811/29064971] boot 2024-05-03T20:29:24.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:24 smithi204 ceph-mon[33595]: osdmap e27: 4 total, 4 up, 4 in 2024-05-03T20:29:24.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:24 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T20:29:24.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:24 smithi181 ceph-mon[29801]: purged_snaps scrub starts 2024-05-03T20:29:24.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:24 smithi181 ceph-mon[29801]: purged_snaps scrub ok 2024-05-03T20:29:24.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:24 smithi181 ceph-mon[29801]: osdmap e26: 4 total, 3 up, 4 in 2024-05-03T20:29:24.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:24 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T20:29:24.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:24 smithi181 ceph-mon[29801]: osd.3 [v2:172.21.15.181:6810/29064971,v1:172.21.15.181:6811/29064971] boot 2024-05-03T20:29:24.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:24 smithi181 ceph-mon[29801]: osdmap e27: 4 total, 4 up, 4 in 2024-05-03T20:29:24.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:24 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T20:29:25.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.306+0000 7f31439f3700 1 -- 172.21.15.196:0/4070840208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f313c103f90 msgr2=0x7f313c106380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:29:25.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.306+0000 7f31439f3700 1 --2- 172.21.15.196:0/4070840208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f313c103f90 0x7f313c106380 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3138009990 tx=0x7f3138009ca0 comp rx=0 tx=0).stop 2024-05-03T20:29:25.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.306+0000 7f31439f3700 1 -- 172.21.15.196:0/4070840208 shutdown_connections 2024-05-03T20:29:25.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.306+0000 7f31439f3700 1 --2- 172.21.15.196:0/4070840208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f313c1091f0 0x7f313c10b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:29:25.308 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.306+0000 7f31439f3700 1 --2- 172.21.15.196:0/4070840208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f313c103f90 0x7f313c106380 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:29:25.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.306+0000 7f31439f3700 1 --2- 172.21.15.196:0/4070840208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f313c1068c0 0x7f313c108cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:29:25.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.306+0000 7f31439f3700 1 -- 172.21.15.196:0/4070840208 >> 172.21.15.196:0/4070840208 conn(0x7f313c0fd9a0 msgr2=0x7f313c0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:29:25.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.307+0000 7f31439f3700 1 -- 172.21.15.196:0/4070840208 shutdown_connections 2024-05-03T20:29:25.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.307+0000 7f31439f3700 1 -- 172.21.15.196:0/4070840208 wait complete. 2024-05-03T20:29:25.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.307+0000 7f31439f3700 1 Processor -- start 2024-05-03T20:29:25.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.307+0000 7f31439f3700 1 -- start start 2024-05-03T20:29:25.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.307+0000 7f31439f3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f313c103f90 0x7f313c103850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:29:25.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.307+0000 7f31439f3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f313c1068c0 0x7f313c101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:29:25.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.307+0000 7f31439f3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f313c1091f0 0x7f313c102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:29:25.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.307+0000 7f31439f3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f313c10f010 con 0x7f313c1068c0 2024-05-03T20:29:25.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.307+0000 7f31439f3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f313c10ee90 con 0x7f313c1091f0 2024-05-03T20:29:25.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.307+0000 7f31439f3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f313c10f190 con 0x7f313c103f90 2024-05-03T20:29:25.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.308+0000 7f3141f90700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f313c1091f0 0x7f313c102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:29:25.309 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.308+0000 7f3141f90700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f313c1091f0 0x7f313c102480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:53248/0 (socket says 172.21.15.196:53248) 2024-05-03T20:29:25.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.308+0000 7f3141f90700 1 -- 172.21.15.196:0/4038667108 learned_addr learned my addr 172.21.15.196:0/4038667108 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-05-03T20:29:25.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.308+0000 7f3140f8e700 1 --2- 172.21.15.196:0/4038667108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f313c1068c0 0x7f313c101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:29:25.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.308+0000 7f314178f700 1 --2- 172.21.15.196:0/4038667108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f313c103f90 0x7f313c103850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:29:25.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.308+0000 7f3141f90700 1 -- 172.21.15.196:0/4038667108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f313c103f90 msgr2=0x7f313c103850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:29:25.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.308+0000 7f3141f90700 1 --2- 172.21.15.196:0/4038667108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f313c103f90 0x7f313c103850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:29:25.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.308+0000 7f3141f90700 1 -- 172.21.15.196:0/4038667108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f313c1068c0 msgr2=0x7f313c101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:29:25.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.308+0000 7f3141f90700 1 --2- 172.21.15.196:0/4038667108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f313c1068c0 0x7f313c101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:29:25.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.308+0000 7f3141f90700 1 -- 172.21.15.196:0/4038667108 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313c102c90 con 0x7f313c1091f0 2024-05-03T20:29:25.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.308+0000 7f3141f90700 1 --2- 172.21.15.196:0/4038667108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f313c1091f0 0x7f313c102480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f31340094f0 tx=0x7f313400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:29:25.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.308+0000 7f31327fc700 1 -- 172.21.15.196:0/4038667108 <== mon.2 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3134004db0 con 0x7f313c1091f0 2024-05-03T20:29:25.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.308+0000 7f31327fc700 1 -- 172.21.15.196:0/4038667108 <== mon.2 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3134009e20 con 0x7f313c1091f0 2024-05-03T20:29:25.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.308+0000 7f31439f3700 1 -- 172.21.15.196:0/4038667108 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313c1a37a0 con 0x7f313c1091f0 2024-05-03T20:29:25.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.308+0000 7f31327fc700 1 -- 172.21.15.196:0/4038667108 <== mon.2 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3134015400 con 0x7f313c1091f0 2024-05-03T20:29:25.310 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.308+0000 7f31439f3700 1 -- 172.21.15.196:0/4038667108 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f313c1a3cb0 con 0x7f313c1091f0 2024-05-03T20:29:25.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.309+0000 7f31439f3700 1 -- 172.21.15.196:0/4038667108 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f313c06a190 con 0x7f313c1091f0 2024-05-03T20:29:25.311 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.310+0000 7f31327fc700 1 -- 172.21.15.196:0/4038667108 <== mon.2 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f3134005260 con 0x7f313c1091f0 2024-05-03T20:29:25.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.311+0000 7f31327fc700 1 --2- 172.21.15.196:0/4038667108 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f31280b2880 0x7f31280b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:29:25.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.311+0000 7f31327fc700 1 -- 172.21.15.196:0/4038667108 <== mon.2 v2:172.21.15.196:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7f31340e09b0 con 0x7f313c1091f0 2024-05-03T20:29:25.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.311+0000 7f314178f700 1 --2- 172.21.15.196:0/4038667108 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f31280b2880 0x7f31280b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:29:25.312 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.312+0000 7f314178f700 1 --2- 172.21.15.196:0/4038667108 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f31280b2880 0x7f31280b4d40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f313800b5c0 tx=0x7f3138005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:29:25.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.316+0000 7f31327fc700 1 -- 172.21.15.196:0/4038667108 <== mon.2 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3134010070 con 0x7f313c1091f0 2024-05-03T20:29:25.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:25 smithi196 ceph-mon[32858]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:25.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:25 smithi196 ceph-mon[32858]: Detected new or changed devices on smithi181 2024-05-03T20:29:25.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:25.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:25.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:29:25.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:29:25.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:25 smithi196 ceph-mon[32858]: Adjusting osd_memory_target on smithi181 to 8544M 2024-05-03T20:29:25.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:25.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:25.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:29:25.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:25.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:29:25.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:25.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:29:25.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:25.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:25 smithi196 ceph-mon[32858]: osdmap e28: 4 total, 4 up, 4 in 2024-05-03T20:29:25.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:25 smithi181 ceph-mon[29801]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:25.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:25 smithi181 ceph-mon[29801]: Detected new or changed devices on smithi181 2024-05-03T20:29:25.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:25.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:25.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:29:25.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:29:25.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:25 smithi181 ceph-mon[29801]: Adjusting osd_memory_target on smithi181 to 8544M 2024-05-03T20:29:25.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:25.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:25.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:29:25.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:25.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:29:25.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:25.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:29:25.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:25.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:25 smithi181 ceph-mon[29801]: osdmap e28: 4 total, 4 up, 4 in 2024-05-03T20:29:25.529 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:25.529+0000 7f31439f3700 1 -- 172.21.15.196:0/4038667108 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f313c0638e0 con 0x7f31280b2880 2024-05-03T20:29:25.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:25 smithi204 ceph-mon[33595]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-05-03T20:29:25.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:25 smithi204 ceph-mon[33595]: Detected new or changed devices on smithi181 2024-05-03T20:29:25.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:25.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:25.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:29:25.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:29:25.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:25 smithi204 ceph-mon[33595]: Adjusting osd_memory_target on smithi181 to 8544M 2024-05-03T20:29:25.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:25.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:25.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:29:25.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:25.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:29:25.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:25.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:29:25.615 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:25.615 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:25 smithi204 ceph-mon[33595]: osdmap e28: 4 total, 4 up, 4 in 2024-05-03T20:29:26.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:26 smithi181 ceph-mon[29801]: osdmap e29: 4 total, 4 up, 4 in 2024-05-03T20:29:26.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:26 smithi181 ceph-mon[29801]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:29:26.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:26 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:29:26.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:26 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:29:26.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:26 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:26.825 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:26 smithi196 ceph-mon[32858]: osdmap e29: 4 total, 4 up, 4 in 2024-05-03T20:29:26.825 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:26 smithi196 ceph-mon[32858]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:29:26.825 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:26 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:29:26.825 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:26 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:29:26.825 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:26 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:26.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:26 smithi204 ceph-mon[33595]: osdmap e29: 4 total, 4 up, 4 in 2024-05-03T20:29:26.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:26 smithi204 ceph-mon[33595]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:29:26.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:26 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:29:26.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:26 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:29:26.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:26 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:27.453 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:27 smithi196 ceph-mon[32858]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-03T20:29:27.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:27 smithi181 ceph-mon[29801]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-03T20:29:27.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:27 smithi204 ceph-mon[33595]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-03T20:29:28.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:28 smithi181 ceph-mon[29801]: from='client.? 172.21.15.196:0/3512125694' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "550f01ef-1947-4f9f-b759-34e3ea831839"}]: dispatch 2024-05-03T20:29:28.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:28 smithi181 ceph-mon[29801]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "550f01ef-1947-4f9f-b759-34e3ea831839"}]: dispatch 2024-05-03T20:29:28.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:28 smithi181 ceph-mon[29801]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "550f01ef-1947-4f9f-b759-34e3ea831839"}]': finished 2024-05-03T20:29:28.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:28 smithi181 ceph-mon[29801]: osdmap e30: 5 total, 4 up, 5 in 2024-05-03T20:29:28.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:28 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T20:29:28.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:28 smithi196 ceph-mon[32858]: from='client.? 172.21.15.196:0/3512125694' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "550f01ef-1947-4f9f-b759-34e3ea831839"}]: dispatch 2024-05-03T20:29:28.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:28 smithi196 ceph-mon[32858]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "550f01ef-1947-4f9f-b759-34e3ea831839"}]: dispatch 2024-05-03T20:29:28.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:28 smithi196 ceph-mon[32858]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "550f01ef-1947-4f9f-b759-34e3ea831839"}]': finished 2024-05-03T20:29:28.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:28 smithi196 ceph-mon[32858]: osdmap e30: 5 total, 4 up, 5 in 2024-05-03T20:29:28.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:28 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T20:29:28.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:28 smithi204 ceph-mon[33595]: from='client.? 172.21.15.196:0/3512125694' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "550f01ef-1947-4f9f-b759-34e3ea831839"}]: dispatch 2024-05-03T20:29:28.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:28 smithi204 ceph-mon[33595]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "550f01ef-1947-4f9f-b759-34e3ea831839"}]: dispatch 2024-05-03T20:29:28.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:28 smithi204 ceph-mon[33595]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "550f01ef-1947-4f9f-b759-34e3ea831839"}]': finished 2024-05-03T20:29:28.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:28 smithi204 ceph-mon[33595]: osdmap e30: 5 total, 4 up, 5 in 2024-05-03T20:29:28.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:28 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T20:29:29.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:29 smithi181 ceph-mon[29801]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-05-03T20:29:29.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:29 smithi181 ceph-mon[29801]: from='client.? 172.21.15.196:0/1435358172' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:29:29.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:29 smithi196 ceph-mon[32858]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-05-03T20:29:29.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:29 smithi196 ceph-mon[32858]: from='client.? 172.21.15.196:0/1435358172' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:29:29.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:29 smithi204 ceph-mon[33595]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-05-03T20:29:29.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:29 smithi204 ceph-mon[33595]: from='client.? 172.21.15.196:0/1435358172' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:29:31.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:31 smithi181 ceph-mon[29801]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-05-03T20:29:31.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:31 smithi196 ceph-mon[32858]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-05-03T20:29:31.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:31 smithi204 ceph-mon[33595]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-05-03T20:29:33.485 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:33 smithi196 ceph-mon[32858]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-03T20:29:33.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:33 smithi181 ceph-mon[29801]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-03T20:29:33.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:33 smithi204 ceph-mon[33595]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-03T20:29:35.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:35 smithi196 ceph-mon[32858]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-03T20:29:35.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:35 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-03T20:29:35.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:35 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:35.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:35 smithi181 ceph-mon[29801]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-03T20:29:35.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:35 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-03T20:29:35.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:35 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:35.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:35 smithi204 ceph-mon[33595]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-03T20:29:35.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:35 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-03T20:29:35.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:35 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:36.551 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:36 smithi196 ceph-mon[32858]: Deploying daemon osd.4 on smithi196 2024-05-03T20:29:36.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:36 smithi181 ceph-mon[29801]: Deploying daemon osd.4 on smithi196 2024-05-03T20:29:36.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:36 smithi204 ceph-mon[33595]: Deploying daemon osd.4 on smithi196 2024-05-03T20:29:37.678 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:37 smithi196 ceph-mon[32858]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T20:29:37.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:37 smithi181 ceph-mon[29801]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T20:29:37.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:37 smithi204 ceph-mon[33595]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T20:29:39.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:39 smithi181 ceph-mon[29801]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-03T20:29:39.713 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:39 smithi196 ceph-mon[32858]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-03T20:29:39.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:39 smithi204 ceph-mon[33595]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-03T20:29:41.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:41 smithi196 ceph-mon[32858]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-03T20:29:41.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:41 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:41.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:41 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:41.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:41 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:29:41.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:41 smithi204 ceph-mon[33595]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-03T20:29:41.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:41 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:41.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:41 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:41.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:41 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:29:41.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:41 smithi181 ceph-mon[29801]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-03T20:29:41.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:41 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:41.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:41 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:41.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:41 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:29:43.668 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:43 smithi196 ceph-mon[32858]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-03T20:29:43.741 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 4 on host 'smithi196' 2024-05-03T20:29:43.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:43.741+0000 7f31327fc700 1 -- 172.21.15.196:0/4038667108 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f313c0638e0 con 0x7f31280b2880 2024-05-03T20:29:43.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:43.744+0000 7f31439f3700 1 -- 172.21.15.196:0/4038667108 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f31280b2880 msgr2=0x7f31280b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:29:43.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:43.744+0000 7f31439f3700 1 --2- 172.21.15.196:0/4038667108 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f31280b2880 0x7f31280b4d40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f313800b5c0 tx=0x7f3138005fb0 comp rx=0 tx=0).stop 2024-05-03T20:29:43.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:43.744+0000 7f31439f3700 1 -- 172.21.15.196:0/4038667108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f313c1091f0 msgr2=0x7f313c102480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:29:43.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:43.744+0000 7f31439f3700 1 --2- 172.21.15.196:0/4038667108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f313c1091f0 0x7f313c102480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f31340094f0 tx=0x7f313400d3b0 comp rx=0 tx=0).stop 2024-05-03T20:29:43.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:43.744+0000 7f31439f3700 1 -- 172.21.15.196:0/4038667108 shutdown_connections 2024-05-03T20:29:43.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:43.744+0000 7f31439f3700 1 --2- 172.21.15.196:0/4038667108 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f31280b2880 0x7f31280b4d40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:29:43.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:43.744+0000 7f31439f3700 1 --2- 172.21.15.196:0/4038667108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f313c1091f0 0x7f313c102480 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:29:43.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:43.744+0000 7f31439f3700 1 --2- 172.21.15.196:0/4038667108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f313c103f90 0x7f313c103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:29:43.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:43.744+0000 7f31439f3700 1 --2- 172.21.15.196:0/4038667108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f313c1068c0 0x7f313c101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:29:43.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:43.744+0000 7f31439f3700 1 -- 172.21.15.196:0/4038667108 >> 172.21.15.196:0/4038667108 conn(0x7f313c0fd9a0 msgr2=0x7f313c104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:29:43.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:43.744+0000 7f31439f3700 1 -- 172.21.15.196:0/4038667108 shutdown_connections 2024-05-03T20:29:43.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:29:43.744+0000 7f31439f3700 1 -- 172.21.15.196:0/4038667108 wait complete. 2024-05-03T20:29:43.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:43 smithi204 ceph-mon[33595]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-03T20:29:43.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:43 smithi181 ceph-mon[29801]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-03T20:29:44.329 DEBUG:teuthology.orchestra.run.smithi196:osd.4> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.4.service 2024-05-03T20:29:44.332 INFO:tasks.cephadm:Deploying osd.5 on smithi204 with /dev/vg_nvme/lv_3... 2024-05-03T20:29:44.332 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-03T20:29:44.597 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T20:29:44.606 INFO:journalctl@ceph.osd.4.smithi196.stdout:May 03 20:29:44 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-4[44447]: 2024-05-03T20:29:44.444+0000 7fd0bd3357c0 -1 osd.4 0 log_to_monitors true 2024-05-03T20:29:44.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:44 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:44.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:44 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:44.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:44 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:44.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:44 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:44.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:44 smithi181 ceph-mon[29801]: from='osd.4 [v2:172.21.15.196:6808/1145408995,v1:172.21.15.196:6809/1145408995]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T20:29:44.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:44 smithi181 ceph-mon[29801]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T20:29:44.971 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:44 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:44.971 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:44 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:44.971 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:44 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:44.971 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:44 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:44.971 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:44 smithi204 ceph-mon[33595]: from='osd.4 [v2:172.21.15.196:6808/1145408995,v1:172.21.15.196:6809/1145408995]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T20:29:44.971 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:44 smithi204 ceph-mon[33595]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T20:29:45.027 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:44 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:45.027 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:44 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:45.027 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:44 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:45.027 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:44 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:45.027 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:44 smithi196 ceph-mon[32858]: from='osd.4 [v2:172.21.15.196:6808/1145408995,v1:172.21.15.196:6809/1145408995]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T20:29:45.027 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:44 smithi196 ceph-mon[32858]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T20:29:45.827 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:45 smithi204 ceph-mon[33595]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-03T20:29:45.827 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:45 smithi204 ceph-mon[33595]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-03T20:29:45.827 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:45 smithi204 ceph-mon[33595]: osdmap e31: 5 total, 4 up, 5 in 2024-05-03T20:29:45.827 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:45 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T20:29:45.827 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:45 smithi204 ceph-mon[33595]: from='osd.4 [v2:172.21.15.196:6808/1145408995,v1:172.21.15.196:6809/1145408995]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-03T20:29:45.827 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:45 smithi204 ceph-mon[33595]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-03T20:29:45.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:45 smithi196 ceph-mon[32858]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-03T20:29:45.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:45 smithi196 ceph-mon[32858]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-03T20:29:45.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:45 smithi196 ceph-mon[32858]: osdmap e31: 5 total, 4 up, 5 in 2024-05-03T20:29:45.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:45 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T20:29:45.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:45 smithi196 ceph-mon[32858]: from='osd.4 [v2:172.21.15.196:6808/1145408995,v1:172.21.15.196:6809/1145408995]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-03T20:29:45.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:45 smithi196 ceph-mon[32858]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-03T20:29:46.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:45 smithi181 ceph-mon[29801]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-03T20:29:46.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:45 smithi181 ceph-mon[29801]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-03T20:29:46.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:45 smithi181 ceph-mon[29801]: osdmap e31: 5 total, 4 up, 5 in 2024-05-03T20:29:46.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:45 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T20:29:46.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:45 smithi181 ceph-mon[29801]: from='osd.4 [v2:172.21.15.196:6808/1145408995,v1:172.21.15.196:6809/1145408995]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-03T20:29:46.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:45 smithi181 ceph-mon[29801]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-03T20:29:46.357 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-03T20:29:46.357 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-03T20:29:46.357 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-05-03T20:29:46.357 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-05-03T20:29:46.357 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112411 s, 933 MB/s 2024-05-03T20:29:46.357 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-05-03T20:29:46.607 INFO:journalctl@ceph.osd.4.smithi196.stdout:May 03 20:29:46 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-4[44447]: 2024-05-03T20:29:46.432+0000 7fd0b2999700 -1 osd.4 0 waiting for initial osdmap 2024-05-03T20:29:46.607 INFO:journalctl@ceph.osd.4.smithi196.stdout:May 03 20:29:46 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-4[44447]: 2024-05-03T20:29:46.442+0000 7fd0ae78c700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T20:29:47.032 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_3 2024-05-03T20:29:47.276 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T20:29:47.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:47 smithi196 ceph-mon[32858]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-03T20:29:47.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:47 smithi196 ceph-mon[32858]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-03T20:29:47.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:47 smithi196 ceph-mon[32858]: osdmap e32: 5 total, 4 up, 5 in 2024-05-03T20:29:47.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:47 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T20:29:47.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:47 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T20:29:47.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:47 smithi181 ceph-mon[29801]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-03T20:29:47.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:47 smithi181 ceph-mon[29801]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-03T20:29:47.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:47 smithi181 ceph-mon[29801]: osdmap e32: 5 total, 4 up, 5 in 2024-05-03T20:29:47.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:47 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T20:29:47.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:47 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T20:29:47.725 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:47 smithi204 ceph-mon[33595]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-03T20:29:47.725 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:47 smithi204 ceph-mon[33595]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-03T20:29:47.725 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:47 smithi204 ceph-mon[33595]: osdmap e32: 5 total, 4 up, 5 in 2024-05-03T20:29:47.725 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:47 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T20:29:47.725 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:47 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T20:29:48.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:48 smithi181 ceph-mon[29801]: purged_snaps scrub starts 2024-05-03T20:29:48.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:48 smithi181 ceph-mon[29801]: purged_snaps scrub ok 2024-05-03T20:29:48.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:48 smithi181 ceph-mon[29801]: osd.4 [v2:172.21.15.196:6808/1145408995,v1:172.21.15.196:6809/1145408995] boot 2024-05-03T20:29:48.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:48 smithi181 ceph-mon[29801]: osdmap e33: 5 total, 5 up, 5 in 2024-05-03T20:29:48.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T20:29:48.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:48 smithi181 ceph-mon[29801]: Detected new or changed devices on smithi196 2024-05-03T20:29:48.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:48.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:48.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:29:48.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:29:48.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:48 smithi181 ceph-mon[29801]: Adjusting osd_memory_target on smithi196 to 8544M 2024-05-03T20:29:48.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:48.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:48.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:29:48.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:48.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:29:48.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:48.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:29:48.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:48.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:48 smithi204 ceph-mon[33595]: purged_snaps scrub starts 2024-05-03T20:29:48.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:48 smithi204 ceph-mon[33595]: purged_snaps scrub ok 2024-05-03T20:29:48.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:48 smithi204 ceph-mon[33595]: osd.4 [v2:172.21.15.196:6808/1145408995,v1:172.21.15.196:6809/1145408995] boot 2024-05-03T20:29:48.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:48 smithi204 ceph-mon[33595]: osdmap e33: 5 total, 5 up, 5 in 2024-05-03T20:29:48.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T20:29:48.757 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:48 smithi204 ceph-mon[33595]: Detected new or changed devices on smithi196 2024-05-03T20:29:48.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:48.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:48.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:29:48.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:29:48.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:48 smithi204 ceph-mon[33595]: Adjusting osd_memory_target on smithi196 to 8544M 2024-05-03T20:29:48.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:48.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:48.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:29:48.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:48.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:29:48.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:48.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:29:48.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:48.777 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.778+0000 7ff96c349700 1 -- 172.21.15.204:0/941102551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff964103620 msgr2=0x7ff964103a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:29:48.777 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.778+0000 7ff96c349700 1 --2- 172.21.15.204:0/941102551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff964103620 0x7ff964103a80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff954009990 tx=0x7ff954009ca0 comp rx=0 tx=0).stop 2024-05-03T20:29:48.777 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.778+0000 7ff96c349700 1 -- 172.21.15.204:0/941102551 shutdown_connections 2024-05-03T20:29:48.777 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.778+0000 7ff96c349700 1 --2- 172.21.15.204:0/941102551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff964103fc0 0x7ff96410a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:29:48.777 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.778+0000 7ff96c349700 1 --2- 172.21.15.204:0/941102551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff964102420 0x7ff964102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:29:48.777 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.778+0000 7ff96c349700 1 --2- 172.21.15.204:0/941102551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff964103620 0x7ff964103a80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:29:48.777 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.778+0000 7ff96c349700 1 -- 172.21.15.204:0/941102551 >> 172.21.15.204:0/941102551 conn(0x7ff9640fd9c0 msgr2=0x7ff9640ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:29:48.777 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.778+0000 7ff96c349700 1 -- 172.21.15.204:0/941102551 shutdown_connections 2024-05-03T20:29:48.778 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.778+0000 7ff96c349700 1 -- 172.21.15.204:0/941102551 wait complete. 2024-05-03T20:29:48.779 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.778+0000 7ff96c349700 1 Processor -- start 2024-05-03T20:29:48.780 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.779+0000 7ff96c349700 1 -- start start 2024-05-03T20:29:48.780 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.779+0000 7ff96c349700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff964102420 0x7ff964198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:29:48.780 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.779+0000 7ff96c349700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff964103620 0x7ff964198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:29:48.780 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.779+0000 7ff96c349700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff964103fc0 0x7ff96419fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:29:48.780 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.779+0000 7ff96c349700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff96410df10 con 0x7ff964102420 2024-05-03T20:29:48.780 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.779+0000 7ff96c349700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff96410dd90 con 0x7ff964103620 2024-05-03T20:29:48.780 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.779+0000 7ff96c349700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff96410e090 con 0x7ff964103fc0 2024-05-03T20:29:48.780 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.779+0000 7ff96a8e6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff964103fc0 0x7ff96419fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:29:48.780 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.779+0000 7ff96a8e6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff964103fc0 0x7ff96419fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:47258/0 (socket says 172.21.15.204:47258) 2024-05-03T20:29:48.780 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.779+0000 7ff96a8e6700 1 -- 172.21.15.204:0/3625914856 learned_addr learned my addr 172.21.15.204:0/3625914856 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-03T20:29:48.780 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.779+0000 7ff96a0e5700 1 --2- 172.21.15.204:0/3625914856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff964102420 0x7ff964198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:29:48.780 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.779+0000 7ff9698e4700 1 --2- 172.21.15.204:0/3625914856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff964103620 0x7ff964198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:29:48.780 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.779+0000 7ff96a8e6700 1 -- 172.21.15.204:0/3625914856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff964103620 msgr2=0x7ff964198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:29:48.780 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.779+0000 7ff96a8e6700 1 --2- 172.21.15.204:0/3625914856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff964103620 0x7ff964198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:29:48.781 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.779+0000 7ff96a8e6700 1 -- 172.21.15.204:0/3625914856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff964102420 msgr2=0x7ff964198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:29:48.781 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.779+0000 7ff96a8e6700 1 --2- 172.21.15.204:0/3625914856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff964102420 0x7ff964198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:29:48.781 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.779+0000 7ff96a8e6700 1 -- 172.21.15.204:0/3625914856 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9641a01e0 con 0x7ff964103fc0 2024-05-03T20:29:48.781 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.779+0000 7ff96a8e6700 1 --2- 172.21.15.204:0/3625914856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff964103fc0 0x7ff96419fb40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff95c007f40 tx=0x7ff95c00f5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:29:48.781 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.779+0000 7ff95b7fe700 1 -- 172.21.15.204:0/3625914856 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c016030 con 0x7ff964103fc0 2024-05-03T20:29:48.781 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.780+0000 7ff96c349700 1 -- 172.21.15.204:0/3625914856 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96406bc30 con 0x7ff964103fc0 2024-05-03T20:29:48.781 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.780+0000 7ff95b7fe700 1 -- 172.21.15.204:0/3625914856 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff95c00fd30 con 0x7ff964103fc0 2024-05-03T20:29:48.781 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.780+0000 7ff96c349700 1 -- 172.21.15.204:0/3625914856 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff96406c1c0 con 0x7ff964103fc0 2024-05-03T20:29:48.781 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.780+0000 7ff95b7fe700 1 -- 172.21.15.204:0/3625914856 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c012e10 con 0x7ff964103fc0 2024-05-03T20:29:48.781 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.780+0000 7ff96c349700 1 -- 172.21.15.204:0/3625914856 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff96406a190 con 0x7ff964103fc0 2024-05-03T20:29:48.782 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.783+0000 7ff95b7fe700 1 -- 172.21.15.204:0/3625914856 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7ff95c012460 con 0x7ff964103fc0 2024-05-03T20:29:48.782 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.783+0000 7ff95b7fe700 1 --2- 172.21.15.204:0/3625914856 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7ff9500b2880 0x7ff9500b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:29:48.782 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.783+0000 7ff95b7fe700 1 -- 172.21.15.204:0/3625914856 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4101+0+0 (secure 0 0 0) 0x7ff95c0e37f0 con 0x7ff964103fc0 2024-05-03T20:29:48.783 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.784+0000 7ff96a0e5700 1 --2- 172.21.15.204:0/3625914856 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7ff9500b2880 0x7ff9500b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:29:48.783 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.784+0000 7ff96a0e5700 1 --2- 172.21.15.204:0/3625914856 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7ff9500b2880 0x7ff9500b4d40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff964103480 tx=0x7ff960009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:29:48.786 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.787+0000 7ff95b7fe700 1 -- 172.21.15.204:0/3625914856 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff95c0ac580 con 0x7ff964103fc0 2024-05-03T20:29:48.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:48 smithi196 ceph-mon[32858]: purged_snaps scrub starts 2024-05-03T20:29:48.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:48 smithi196 ceph-mon[32858]: purged_snaps scrub ok 2024-05-03T20:29:48.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:48 smithi196 ceph-mon[32858]: osd.4 [v2:172.21.15.196:6808/1145408995,v1:172.21.15.196:6809/1145408995] boot 2024-05-03T20:29:48.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:48 smithi196 ceph-mon[32858]: osdmap e33: 5 total, 5 up, 5 in 2024-05-03T20:29:48.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T20:29:48.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:48 smithi196 ceph-mon[32858]: Detected new or changed devices on smithi196 2024-05-03T20:29:48.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:48.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:48.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:29:48.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:29:48.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:48 smithi196 ceph-mon[32858]: Adjusting osd_memory_target on smithi196 to 8544M 2024-05-03T20:29:48.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:48.859 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:48.859 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:29:48.859 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:48.859 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:29:48.859 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:48.859 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:29:48.859 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:29:48.998 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:29:48.999+0000 7ff96c349700 1 -- 172.21.15.204:0/3625914856 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff96406c4e0 con 0x7ff9500b2880 2024-05-03T20:29:49.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:49 smithi204 ceph-mon[33595]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:29:49.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:49 smithi204 ceph-mon[33595]: osdmap e34: 5 total, 5 up, 5 in 2024-05-03T20:29:49.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:49 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:29:49.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:49 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:29:49.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:49 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:50.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:49 smithi196 ceph-mon[32858]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:29:50.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:49 smithi196 ceph-mon[32858]: osdmap e34: 5 total, 5 up, 5 in 2024-05-03T20:29:50.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:49 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:29:50.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:49 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:29:50.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:49 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:50.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:49 smithi181 ceph-mon[29801]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:29:50.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:49 smithi181 ceph-mon[29801]: osdmap e34: 5 total, 5 up, 5 in 2024-05-03T20:29:50.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:49 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:29:50.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:49 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:29:50.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:49 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:29:50.833 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:50 smithi204 ceph-mon[33595]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:29:51.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:50 smithi196 ceph-mon[32858]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:29:51.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:50 smithi181 ceph-mon[29801]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:29:51.982 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:51 smithi204 ceph-mon[33595]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:29:51.982 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:51 smithi204 ceph-mon[33595]: from='client.? 172.21.15.204:0/2389962616' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b74cb3a-f39a-4131-8793-1d5286856aa6"}]: dispatch 2024-05-03T20:29:51.983 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:51 smithi204 ceph-mon[33595]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b74cb3a-f39a-4131-8793-1d5286856aa6"}]: dispatch 2024-05-03T20:29:51.983 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:51 smithi204 ceph-mon[33595]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b74cb3a-f39a-4131-8793-1d5286856aa6"}]': finished 2024-05-03T20:29:51.983 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:51 smithi204 ceph-mon[33595]: osdmap e35: 6 total, 5 up, 6 in 2024-05-03T20:29:51.983 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:51 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T20:29:52.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:51 smithi196 ceph-mon[32858]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:29:52.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:51 smithi196 ceph-mon[32858]: from='client.? 172.21.15.204:0/2389962616' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b74cb3a-f39a-4131-8793-1d5286856aa6"}]: dispatch 2024-05-03T20:29:52.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:51 smithi196 ceph-mon[32858]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b74cb3a-f39a-4131-8793-1d5286856aa6"}]: dispatch 2024-05-03T20:29:52.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:51 smithi196 ceph-mon[32858]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b74cb3a-f39a-4131-8793-1d5286856aa6"}]': finished 2024-05-03T20:29:52.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:51 smithi196 ceph-mon[32858]: osdmap e35: 6 total, 5 up, 6 in 2024-05-03T20:29:52.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:51 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T20:29:52.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:51 smithi181 ceph-mon[29801]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:29:52.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:51 smithi181 ceph-mon[29801]: from='client.? 172.21.15.204:0/2389962616' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b74cb3a-f39a-4131-8793-1d5286856aa6"}]: dispatch 2024-05-03T20:29:52.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:51 smithi181 ceph-mon[29801]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b74cb3a-f39a-4131-8793-1d5286856aa6"}]: dispatch 2024-05-03T20:29:52.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:51 smithi181 ceph-mon[29801]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b74cb3a-f39a-4131-8793-1d5286856aa6"}]': finished 2024-05-03T20:29:52.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:51 smithi181 ceph-mon[29801]: osdmap e35: 6 total, 5 up, 6 in 2024-05-03T20:29:52.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:51 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T20:29:53.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:52 smithi196 ceph-mon[32858]: from='client.? 172.21.15.204:0/4225212363' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:29:53.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:52 smithi204 ceph-mon[33595]: from='client.? 172.21.15.204:0/4225212363' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:29:53.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:52 smithi181 ceph-mon[29801]: from='client.? 172.21.15.204:0/4225212363' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:29:54.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:53 smithi196 ceph-mon[32858]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:29:54.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:53 smithi204 ceph-mon[33595]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:29:54.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:53 smithi181 ceph-mon[29801]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:29:55.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:54 smithi196 ceph-mon[32858]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:29:55.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:54 smithi204 ceph-mon[33595]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:29:55.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:54 smithi181 ceph-mon[29801]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:29:57.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:56 smithi204 ceph-mon[33595]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:29:57.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:56 smithi181 ceph-mon[29801]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:29:57.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:56 smithi196 ceph-mon[32858]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:29:59.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:58 smithi204 ceph-mon[33595]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:29:59.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:58 smithi181 ceph-mon[29801]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:29:59.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:58 smithi196 ceph-mon[32858]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:30:00.080 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:59 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-03T20:30:00.080 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:59 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:00.080 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:29:59 smithi204 ceph-mon[33595]: Deploying daemon osd.5 on smithi204 2024-05-03T20:30:00.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:59 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-03T20:30:00.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:59 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:00.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:29:59 smithi181 ceph-mon[29801]: Deploying daemon osd.5 on smithi204 2024-05-03T20:30:00.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:59 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-03T20:30:00.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:59 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:00.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:29:59 smithi196 ceph-mon[32858]: Deploying daemon osd.5 on smithi204 2024-05-03T20:30:01.076 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:00 smithi204 ceph-mon[33595]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:30:01.076 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:00 smithi204 ceph-mon[33595]: overall HEALTH_OK 2024-05-03T20:30:01.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:00 smithi181 ceph-mon[29801]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:30:01.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:00 smithi181 ceph-mon[29801]: overall HEALTH_OK 2024-05-03T20:30:01.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:00 smithi196 ceph-mon[32858]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:30:01.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:00 smithi196 ceph-mon[32858]: overall HEALTH_OK 2024-05-03T20:30:02.970 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:02 smithi204 ceph-mon[33595]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:30:03.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:02 smithi181 ceph-mon[29801]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:30:03.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:02 smithi196 ceph-mon[32858]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:30:05.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:05 smithi196 ceph-mon[32858]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:30:05.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:05 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:05.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:05 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:05.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:05 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:30:05.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:05 smithi204 ceph-mon[33595]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:30:05.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:05 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:05.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:05 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:05.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:05 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:30:05.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:05 smithi181 ceph-mon[29801]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:30:05.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:05 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:05.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:05 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:05.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:05 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:30:07.834 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:07 smithi204 ceph-mon[33595]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:30:07.834 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:07.834 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:07.834 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:07.834 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:30:07.834 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:07.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:07 smithi196 ceph-mon[32858]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:30:07.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:07.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:07.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:07.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:30:07.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:07.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:07 smithi181 ceph-mon[29801]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:30:07.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:07.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:07.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:07.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:30:07.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:08.218 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:30:08.218+0000 7ff95b7fe700 1 -- 172.21.15.204:0/3625914856 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff96406c4e0 con 0x7ff9500b2880 2024-05-03T20:30:08.218 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 5 on host 'smithi204' 2024-05-03T20:30:08.220 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:30:08.221+0000 7ff96c349700 1 -- 172.21.15.204:0/3625914856 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7ff9500b2880 msgr2=0x7ff9500b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:30:08.220 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:30:08.221+0000 7ff96c349700 1 --2- 172.21.15.204:0/3625914856 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7ff9500b2880 0x7ff9500b4d40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff964103480 tx=0x7ff960009260 comp rx=0 tx=0).stop 2024-05-03T20:30:08.220 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:30:08.221+0000 7ff96c349700 1 -- 172.21.15.204:0/3625914856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff964103fc0 msgr2=0x7ff96419fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:30:08.220 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:30:08.221+0000 7ff96c349700 1 --2- 172.21.15.204:0/3625914856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff964103fc0 0x7ff96419fb40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff95c007f40 tx=0x7ff95c00f5c0 comp rx=0 tx=0).stop 2024-05-03T20:30:08.220 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:30:08.221+0000 7ff96c349700 1 -- 172.21.15.204:0/3625914856 shutdown_connections 2024-05-03T20:30:08.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:30:08.221+0000 7ff96c349700 1 --2- 172.21.15.204:0/3625914856 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7ff9500b2880 0x7ff9500b4d40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:30:08.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:30:08.221+0000 7ff96c349700 1 --2- 172.21.15.204:0/3625914856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff964103fc0 0x7ff96419fb40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:30:08.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:30:08.221+0000 7ff96c349700 1 --2- 172.21.15.204:0/3625914856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff964102420 0x7ff964198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:30:08.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:30:08.221+0000 7ff96c349700 1 --2- 172.21.15.204:0/3625914856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff964103620 0x7ff964198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:30:08.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:30:08.221+0000 7ff96c349700 1 -- 172.21.15.204:0/3625914856 >> 172.21.15.204:0/3625914856 conn(0x7ff9640fd9c0 msgr2=0x7ff9641069a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:30:08.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:30:08.221+0000 7ff96c349700 1 -- 172.21.15.204:0/3625914856 shutdown_connections 2024-05-03T20:30:08.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:30:08.221+0000 7ff96c349700 1 -- 172.21.15.204:0/3625914856 wait complete. 2024-05-03T20:30:08.822 DEBUG:teuthology.orchestra.run.smithi204:osd.5> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.5.service 2024-05-03T20:30:08.825 INFO:tasks.cephadm:Deploying osd.6 on smithi181 with /dev/vg_nvme/lv_2... 2024-05-03T20:30:08.825 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-03T20:30:09.074 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:30:09.428 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:09 smithi181 ceph-mon[29801]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:30:09.428 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:09 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:09.428 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:09 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:09.428 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:09 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:30:09.428 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:09 smithi181 ceph-mon[29801]: from='osd.5 [v2:172.21.15.204:6808/855680559,v1:172.21.15.204:6809/855680559]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T20:30:09.428 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:09 smithi181 ceph-mon[29801]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T20:30:09.569 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:09 smithi204 ceph-mon[33595]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:30:09.569 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:09 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:09.569 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:09 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:09.569 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:09 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:30:09.569 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:09 smithi204 ceph-mon[33595]: from='osd.5 [v2:172.21.15.204:6808/855680559,v1:172.21.15.204:6809/855680559]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T20:30:09.569 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:09 smithi204 ceph-mon[33595]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T20:30:09.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:09 smithi196 ceph-mon[32858]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:30:09.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:09 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:09.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:09 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:09.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:09 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:30:09.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:09 smithi196 ceph-mon[32858]: from='osd.5 [v2:172.21.15.204:6808/855680559,v1:172.21.15.204:6809/855680559]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T20:30:09.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:09 smithi196 ceph-mon[32858]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T20:30:10.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:10 smithi181 ceph-mon[29801]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-03T20:30:10.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:10 smithi181 ceph-mon[29801]: osdmap e36: 6 total, 5 up, 6 in 2024-05-03T20:30:10.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:10 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T20:30:10.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:10 smithi181 ceph-mon[29801]: from='osd.5 [v2:172.21.15.204:6808/855680559,v1:172.21.15.204:6809/855680559]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-03T20:30:10.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:10 smithi181 ceph-mon[29801]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-03T20:30:10.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:10 smithi196 ceph-mon[32858]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-03T20:30:10.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:10 smithi196 ceph-mon[32858]: osdmap e36: 6 total, 5 up, 6 in 2024-05-03T20:30:10.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:10 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T20:30:10.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:10 smithi196 ceph-mon[32858]: from='osd.5 [v2:172.21.15.204:6808/855680559,v1:172.21.15.204:6809/855680559]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-03T20:30:10.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:10 smithi196 ceph-mon[32858]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-03T20:30:10.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:10 smithi204 ceph-mon[33595]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-03T20:30:10.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:10 smithi204 ceph-mon[33595]: osdmap e36: 6 total, 5 up, 6 in 2024-05-03T20:30:10.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:10 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T20:30:10.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:10 smithi204 ceph-mon[33595]: from='osd.5 [v2:172.21.15.204:6808/855680559,v1:172.21.15.204:6809/855680559]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-03T20:30:10.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:10 smithi204 ceph-mon[33595]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-03T20:30:10.614 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:30:10 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:30:10.193+0000 7f48be3bf700 -1 osd.5 0 waiting for initial osdmap 2024-05-03T20:30:10.614 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:30:10 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:30:10.200+0000 7f48ba1b2700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T20:30:11.073 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-03T20:30:11.074 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-03T20:30:11.074 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2024-05-03T20:30:11.074 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2024-05-03T20:30:11.074 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114222 s, 918 MB/s 2024-05-03T20:30:11.074 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2024-05-03T20:30:11.382 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:11 smithi181 ceph-mon[29801]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:30:11.383 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:11 smithi181 ceph-mon[29801]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-03T20:30:11.383 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:11 smithi181 ceph-mon[29801]: osdmap e37: 6 total, 5 up, 6 in 2024-05-03T20:30:11.383 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:11 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T20:30:11.383 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:11 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T20:30:11.484 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:11 smithi204 ceph-mon[33595]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:30:11.484 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:11 smithi204 ceph-mon[33595]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-03T20:30:11.485 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:11 smithi204 ceph-mon[33595]: osdmap e37: 6 total, 5 up, 6 in 2024-05-03T20:30:11.485 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:11 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T20:30:11.485 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:11 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T20:30:11.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:11 smithi196 ceph-mon[32858]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:30:11.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:11 smithi196 ceph-mon[32858]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-03T20:30:11.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:11 smithi196 ceph-mon[32858]: osdmap e37: 6 total, 5 up, 6 in 2024-05-03T20:30:11.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:11 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T20:30:11.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:11 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T20:30:11.735 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_2 2024-05-03T20:30:11.975 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:30:12.351 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:12 smithi181 ceph-mon[29801]: purged_snaps scrub starts 2024-05-03T20:30:12.351 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:12 smithi181 ceph-mon[29801]: purged_snaps scrub ok 2024-05-03T20:30:12.351 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T20:30:12.351 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:12 smithi181 ceph-mon[29801]: osd.5 [v2:172.21.15.204:6808/855680559,v1:172.21.15.204:6809/855680559] boot 2024-05-03T20:30:12.351 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:12 smithi181 ceph-mon[29801]: osdmap e38: 6 total, 6 up, 6 in 2024-05-03T20:30:12.351 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T20:30:12.351 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:12 smithi181 ceph-mon[29801]: Detected new or changed devices on smithi204 2024-05-03T20:30:12.351 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:12.351 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:12.351 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:30:12.352 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:30:12.352 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:12 smithi181 ceph-mon[29801]: Adjusting osd_memory_target on smithi204 to 8544M 2024-05-03T20:30:12.352 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:12.352 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:12.352 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:30:12.352 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:12.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:12 smithi196 ceph-mon[32858]: purged_snaps scrub starts 2024-05-03T20:30:12.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:12 smithi196 ceph-mon[32858]: purged_snaps scrub ok 2024-05-03T20:30:12.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T20:30:12.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:12 smithi196 ceph-mon[32858]: osd.5 [v2:172.21.15.204:6808/855680559,v1:172.21.15.204:6809/855680559] boot 2024-05-03T20:30:12.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:12 smithi196 ceph-mon[32858]: osdmap e38: 6 total, 6 up, 6 in 2024-05-03T20:30:12.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T20:30:12.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:12 smithi196 ceph-mon[32858]: Detected new or changed devices on smithi204 2024-05-03T20:30:12.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:12.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:12.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:30:12.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:30:12.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:12 smithi196 ceph-mon[32858]: Adjusting osd_memory_target on smithi204 to 8544M 2024-05-03T20:30:12.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:12.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:12.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:30:12.608 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:12.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:12 smithi204 ceph-mon[33595]: purged_snaps scrub starts 2024-05-03T20:30:12.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:12 smithi204 ceph-mon[33595]: purged_snaps scrub ok 2024-05-03T20:30:12.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T20:30:12.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:12 smithi204 ceph-mon[33595]: osd.5 [v2:172.21.15.204:6808/855680559,v1:172.21.15.204:6809/855680559] boot 2024-05-03T20:30:12.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:12 smithi204 ceph-mon[33595]: osdmap e38: 6 total, 6 up, 6 in 2024-05-03T20:30:12.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T20:30:12.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:12 smithi204 ceph-mon[33595]: Detected new or changed devices on smithi204 2024-05-03T20:30:12.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:12.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:12.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:30:12.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:30:12.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:12 smithi204 ceph-mon[33595]: Adjusting osd_memory_target on smithi204 to 8544M 2024-05-03T20:30:12.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:12.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:12.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:30:12.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:13.322 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:13 smithi181 ceph-mon[29801]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:30:13.323 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:13 smithi181 ceph-mon[29801]: osdmap e39: 6 total, 6 up, 6 in 2024-05-03T20:30:13.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:13 smithi204 ceph-mon[33595]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:30:13.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:13 smithi204 ceph-mon[33595]: osdmap e39: 6 total, 6 up, 6 in 2024-05-03T20:30:13.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.561+0000 7f1f37ce8700 1 -- 172.21.15.181:0/2360188944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f30102e40 msgr2=0x7f1f301032a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:30:13.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.561+0000 7f1f37ce8700 1 --2- 172.21.15.181:0/2360188944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f30102e40 0x7f1f301032a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1f200099c0 tx=0x7f1f20009cd0 comp rx=0 tx=0).stop 2024-05-03T20:30:13.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.561+0000 7f1f37ce8700 1 -- 172.21.15.181:0/2360188944 shutdown_connections 2024-05-03T20:30:13.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.561+0000 7f1f37ce8700 1 --2- 172.21.15.181:0/2360188944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f301037e0 0x7f1f3010a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:30:13.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.561+0000 7f1f37ce8700 1 --2- 172.21.15.181:0/2360188944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f30101c40 0x7f1f30102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:30:13.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.561+0000 7f1f37ce8700 1 --2- 172.21.15.181:0/2360188944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f30102e40 0x7f1f301032a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:30:13.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.561+0000 7f1f37ce8700 1 -- 172.21.15.181:0/2360188944 >> 172.21.15.181:0/2360188944 conn(0x7f1f300fd1e0 msgr2=0x7f1f300ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:30:13.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.561+0000 7f1f37ce8700 1 -- 172.21.15.181:0/2360188944 shutdown_connections 2024-05-03T20:30:13.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.561+0000 7f1f37ce8700 1 -- 172.21.15.181:0/2360188944 wait complete. 2024-05-03T20:30:13.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.561+0000 7f1f37ce8700 1 Processor -- start 2024-05-03T20:30:13.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.562+0000 7f1f37ce8700 1 -- start start 2024-05-03T20:30:13.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.562+0000 7f1f37ce8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f30101c40 0x7f1f30199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:30:13.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.562+0000 7f1f37ce8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f30102e40 0x7f1f30199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:30:13.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.562+0000 7f1f37ce8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f301037e0 0x7f1f301a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:30:13.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.562+0000 7f1f37ce8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3010e000 con 0x7f1f30101c40 2024-05-03T20:30:13.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.562+0000 7f1f37ce8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3010de80 con 0x7f1f301037e0 2024-05-03T20:30:13.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.562+0000 7f1f37ce8700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3010e180 con 0x7f1f30102e40 2024-05-03T20:30:13.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.562+0000 7f1f35a84700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f30101c40 0x7f1f30199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:30:13.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.562+0000 7f1f35a84700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f30101c40 0x7f1f30199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39142/0 (socket says 172.21.15.181:39142) 2024-05-03T20:30:13.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.562+0000 7f1f35a84700 1 -- 172.21.15.181:0/1919422444 learned_addr learned my addr 172.21.15.181:0/1919422444 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:30:13.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.562+0000 7f1f35283700 1 --2- 172.21.15.181:0/1919422444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f30102e40 0x7f1f30199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:30:13.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.562+0000 7f1f36285700 1 --2- 172.21.15.181:0/1919422444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f301037e0 0x7f1f301a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:30:13.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.562+0000 7f1f35a84700 1 -- 172.21.15.181:0/1919422444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f30102e40 msgr2=0x7f1f30199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:30:13.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.562+0000 7f1f35a84700 1 --2- 172.21.15.181:0/1919422444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f30102e40 0x7f1f30199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:30:13.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.562+0000 7f1f35a84700 1 -- 172.21.15.181:0/1919422444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f301037e0 msgr2=0x7f1f301a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:30:13.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.562+0000 7f1f35a84700 1 --2- 172.21.15.181:0/1919422444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f301037e0 0x7f1f301a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:30:13.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.562+0000 7f1f35a84700 1 -- 172.21.15.181:0/1919422444 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f301a13c0 con 0x7f1f30101c40 2024-05-03T20:30:13.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.562+0000 7f1f36285700 1 --2- 172.21.15.181:0/1919422444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f301037e0 0x7f1f301a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:30:13.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.562+0000 7f1f35a84700 1 --2- 172.21.15.181:0/1919422444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f30101c40 0x7f1f30199760 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00b530 tx=0x7f1f2c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:30:13.565 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.562+0000 7f1f26ffd700 1 -- 172.21.15.181:0/1919422444 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c015020 con 0x7f1f30101c40 2024-05-03T20:30:13.565 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.562+0000 7f1f26ffd700 1 -- 172.21.15.181:0/1919422444 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f2c011ce0 con 0x7f1f30101c40 2024-05-03T20:30:13.565 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.562+0000 7f1f37ce8700 1 -- 172.21.15.181:0/1919422444 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f301a16b0 con 0x7f1f30101c40 2024-05-03T20:30:13.565 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.562+0000 7f1f26ffd700 1 -- 172.21.15.181:0/1919422444 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c010d60 con 0x7f1f30101c40 2024-05-03T20:30:13.565 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.563+0000 7f1f37ce8700 1 -- 172.21.15.181:0/1919422444 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f301a1bf0 con 0x7f1f30101c40 2024-05-03T20:30:13.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.563+0000 7f1f37ce8700 1 -- 172.21.15.181:0/1919422444 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f3006a190 con 0x7f1f30101c40 2024-05-03T20:30:13.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.565+0000 7f1f26ffd700 1 -- 172.21.15.181:0/1919422444 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f1f2c018020 con 0x7f1f30101c40 2024-05-03T20:30:13.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.566+0000 7f1f26ffd700 1 --2- 172.21.15.181:0/1919422444 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f1f1c0b2880 0x7f1f1c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:30:13.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.566+0000 7f1f35283700 1 --2- 172.21.15.181:0/1919422444 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f1f1c0b2880 0x7f1f1c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:30:13.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.566+0000 7f1f26ffd700 1 -- 172.21.15.181:0/1919422444 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4533+0+0 (secure 0 0 0) 0x7f1f2c0e10a0 con 0x7f1f30101c40 2024-05-03T20:30:13.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.566+0000 7f1f35283700 1 --2- 172.21.15.181:0/1919422444 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f1f1c0b2880 0x7f1f1c0b4d40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1f20009990 tx=0x7f1f2000b870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:30:13.569 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.569+0000 7f1f26ffd700 1 -- 172.21.15.181:0/1919422444 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1f2c01f020 con 0x7f1f30101c40 2024-05-03T20:30:13.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:13 smithi196 ceph-mon[32858]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T20:30:13.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:13 smithi196 ceph-mon[32858]: osdmap e39: 6 total, 6 up, 6 in 2024-05-03T20:30:13.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:13.784+0000 7f1f37ce8700 1 -- 172.21.15.181:0/1919422444 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f1f300ff3b0 con 0x7f1f1c0b2880 2024-05-03T20:30:14.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:14 smithi181 ceph-mon[29801]: osdmap e40: 6 total, 6 up, 6 in 2024-05-03T20:30:14.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:14 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:30:14.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:14 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:30:14.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:14 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:14.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:14 smithi196 ceph-mon[32858]: osdmap e40: 6 total, 6 up, 6 in 2024-05-03T20:30:14.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:14 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:30:14.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:14 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:30:14.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:14 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:14.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:14 smithi204 ceph-mon[33595]: osdmap e40: 6 total, 6 up, 6 in 2024-05-03T20:30:14.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:14 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:30:14.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:14 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:30:14.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:14 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:15.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:15 smithi181 ceph-mon[29801]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:30:15.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:15 smithi181 ceph-mon[29801]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-03T20:30:15.508 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:15 smithi196 ceph-mon[32858]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:30:15.508 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:15 smithi196 ceph-mon[32858]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-03T20:30:15.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:15 smithi204 ceph-mon[33595]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:30:15.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:15 smithi204 ceph-mon[33595]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-03T20:30:17.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:17 smithi196 ceph-mon[32858]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-03T20:30:17.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:17 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/442516298' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbe19bdb-08eb-40aa-842b-0710963d4497"}]: dispatch 2024-05-03T20:30:17.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:17 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/442516298' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dbe19bdb-08eb-40aa-842b-0710963d4497"}]': finished 2024-05-03T20:30:17.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:17 smithi196 ceph-mon[32858]: osdmap e41: 7 total, 6 up, 7 in 2024-05-03T20:30:17.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:17 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T20:30:17.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:17 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/2878014231' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:30:17.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:17 smithi204 ceph-mon[33595]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-03T20:30:17.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:17 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/442516298' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbe19bdb-08eb-40aa-842b-0710963d4497"}]: dispatch 2024-05-03T20:30:17.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:17 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/442516298' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dbe19bdb-08eb-40aa-842b-0710963d4497"}]': finished 2024-05-03T20:30:17.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:17 smithi204 ceph-mon[33595]: osdmap e41: 7 total, 6 up, 7 in 2024-05-03T20:30:17.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:17 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T20:30:17.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:17 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/2878014231' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:30:17.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:17 smithi181 ceph-mon[29801]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-03T20:30:17.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:17 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/442516298' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbe19bdb-08eb-40aa-842b-0710963d4497"}]: dispatch 2024-05-03T20:30:17.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:17 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/442516298' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dbe19bdb-08eb-40aa-842b-0710963d4497"}]': finished 2024-05-03T20:30:17.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:17 smithi181 ceph-mon[29801]: osdmap e41: 7 total, 6 up, 7 in 2024-05-03T20:30:17.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:17 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T20:30:17.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:17 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/2878014231' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:30:19.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:19 smithi196 ceph-mon[32858]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T20:30:19.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:19 smithi204 ceph-mon[33595]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T20:30:19.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:19 smithi181 ceph-mon[29801]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T20:30:21.564 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:21 smithi181 ceph-mon[29801]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-03T20:30:21.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:21 smithi196 ceph-mon[32858]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-03T20:30:21.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:21 smithi204 ceph-mon[33595]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-03T20:30:23.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:23 smithi196 ceph-mon[32858]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-03T20:30:23.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:23 smithi204 ceph-mon[33595]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-03T20:30:23.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:23 smithi181 ceph-mon[29801]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-03T20:30:24.544 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:24 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-03T20:30:24.544 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:24 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:24.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:24 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-03T20:30:24.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:24 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:24.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:24 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-03T20:30:24.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:24 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:25.433 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:25 smithi181 ceph-mon[29801]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T20:30:25.433 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:25 smithi181 ceph-mon[29801]: Deploying daemon osd.6 on smithi181 2024-05-03T20:30:25.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:25 smithi196 ceph-mon[32858]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T20:30:25.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:25 smithi196 ceph-mon[32858]: Deploying daemon osd.6 on smithi181 2024-05-03T20:30:25.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:25 smithi204 ceph-mon[33595]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T20:30:25.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:25 smithi204 ceph-mon[33595]: Deploying daemon osd.6 on smithi181 2024-05-03T20:30:27.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:27 smithi181 ceph-mon[29801]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T20:30:27.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:27 smithi196 ceph-mon[32858]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T20:30:27.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:27 smithi204 ceph-mon[33595]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T20:30:29.552 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:29 smithi181 ceph-mon[29801]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-05-03T20:30:29.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:29 smithi196 ceph-mon[32858]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-05-03T20:30:29.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:29 smithi204 ceph-mon[33595]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-05-03T20:30:30.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:30 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:30.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:30 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:30.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:30 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:30:30.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:30 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:30.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:30 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:30.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:30 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:30:30.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:30 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:30.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:30 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:30.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:30 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:30:31.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:31 smithi181 ceph-mon[29801]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-03T20:30:31.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:31 smithi196 ceph-mon[32858]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-03T20:30:31.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:31 smithi204 ceph-mon[33595]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-03T20:30:32.690 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:32.690+0000 7f1f26ffd700 1 -- 172.21.15.181:0/1919422444 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1f300ff3b0 con 0x7f1f1c0b2880 2024-05-03T20:30:32.690 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 6 on host 'smithi181' 2024-05-03T20:30:32.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:32.693+0000 7f1f37ce8700 1 -- 172.21.15.181:0/1919422444 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f1f1c0b2880 msgr2=0x7f1f1c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:30:32.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:32.693+0000 7f1f37ce8700 1 --2- 172.21.15.181:0/1919422444 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f1f1c0b2880 0x7f1f1c0b4d40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1f20009990 tx=0x7f1f2000b870 comp rx=0 tx=0).stop 2024-05-03T20:30:32.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:32.693+0000 7f1f37ce8700 1 -- 172.21.15.181:0/1919422444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f30101c40 msgr2=0x7f1f30199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:30:32.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:32.693+0000 7f1f37ce8700 1 --2- 172.21.15.181:0/1919422444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f30101c40 0x7f1f30199760 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00b530 tx=0x7f1f2c00b840 comp rx=0 tx=0).stop 2024-05-03T20:30:32.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:32.693+0000 7f1f37ce8700 1 -- 172.21.15.181:0/1919422444 shutdown_connections 2024-05-03T20:30:32.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:32.693+0000 7f1f37ce8700 1 --2- 172.21.15.181:0/1919422444 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f1f1c0b2880 0x7f1f1c0b4d40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:30:32.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:32.693+0000 7f1f37ce8700 1 --2- 172.21.15.181:0/1919422444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f301037e0 0x7f1f301a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:30:32.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:32.693+0000 7f1f37ce8700 1 --2- 172.21.15.181:0/1919422444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f30101c40 0x7f1f30199760 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:30:32.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:32.693+0000 7f1f37ce8700 1 --2- 172.21.15.181:0/1919422444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f30102e40 0x7f1f30199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:30:32.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:32.693+0000 7f1f37ce8700 1 -- 172.21.15.181:0/1919422444 >> 172.21.15.181:0/1919422444 conn(0x7f1f300fd1e0 msgr2=0x7f1f300feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:30:32.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:32.693+0000 7f1f37ce8700 1 -- 172.21.15.181:0/1919422444 shutdown_connections 2024-05-03T20:30:32.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:30:32.693+0000 7f1f37ce8700 1 -- 172.21.15.181:0/1919422444 wait complete. 2024-05-03T20:30:33.547 DEBUG:teuthology.orchestra.run.smithi181:osd.6> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.6.service 2024-05-03T20:30:33.549 INFO:tasks.cephadm:Deploying osd.7 on smithi196 with /dev/vg_nvme/lv_2... 2024-05-03T20:30:33.549 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-03T20:30:33.798 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T20:30:33.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:33 smithi196 ceph-mon[32858]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-03T20:30:33.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:33 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:33.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:33 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:33.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:33 smithi196 ceph-mon[32858]: from='osd.6 [v2:172.21.15.181:6818/1561969338,v1:172.21.15.181:6819/1561969338]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-03T20:30:33.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:33 smithi181 ceph-mon[29801]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-03T20:30:33.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:33 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:33.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:33 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:33.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:33 smithi181 ceph-mon[29801]: from='osd.6 [v2:172.21.15.181:6818/1561969338,v1:172.21.15.181:6819/1561969338]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-03T20:30:34.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:33 smithi204 ceph-mon[33595]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-03T20:30:34.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:33 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:34.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:33 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:34.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:33 smithi204 ceph-mon[33595]: from='osd.6 [v2:172.21.15.181:6818/1561969338,v1:172.21.15.181:6819/1561969338]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-03T20:30:34.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:34 smithi196 ceph-mon[32858]: from='osd.6 [v2:172.21.15.181:6818/1561969338,v1:172.21.15.181:6819/1561969338]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-03T20:30:34.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:34 smithi196 ceph-mon[32858]: osdmap e42: 7 total, 6 up, 7 in 2024-05-03T20:30:34.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:34 smithi196 ceph-mon[32858]: from='osd.6 [v2:172.21.15.181:6818/1561969338,v1:172.21.15.181:6819/1561969338]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-03T20:30:34.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:34 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T20:30:34.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:34 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:34.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:34 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:34.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:34 smithi181 ceph-mon[29801]: from='osd.6 [v2:172.21.15.181:6818/1561969338,v1:172.21.15.181:6819/1561969338]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-03T20:30:34.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:34 smithi181 ceph-mon[29801]: osdmap e42: 7 total, 6 up, 7 in 2024-05-03T20:30:34.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:34 smithi181 ceph-mon[29801]: from='osd.6 [v2:172.21.15.181:6818/1561969338,v1:172.21.15.181:6819/1561969338]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-03T20:30:34.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:34 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T20:30:34.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:34 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:34.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:34 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:34.964 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:30:34 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:30:34.666+0000 7fa48a804700 -1 osd.6 0 waiting for initial osdmap 2024-05-03T20:30:34.964 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:30:34 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:30:34.681+0000 7fa4865f7700 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T20:30:35.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:34 smithi204 ceph-mon[33595]: from='osd.6 [v2:172.21.15.181:6818/1561969338,v1:172.21.15.181:6819/1561969338]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-03T20:30:35.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:34 smithi204 ceph-mon[33595]: osdmap e42: 7 total, 6 up, 7 in 2024-05-03T20:30:35.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:34 smithi204 ceph-mon[33595]: from='osd.6 [v2:172.21.15.181:6818/1561969338,v1:172.21.15.181:6819/1561969338]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-03T20:30:35.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:34 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T20:30:35.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:34 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:35.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:34 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:35.780 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-03T20:30:35.781 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-03T20:30:35.781 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-05-03T20:30:35.781 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-05-03T20:30:35.781 INFO:teuthology.orchestra.run.smithi196.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114073 s, 919 MB/s 2024-05-03T20:30:35.781 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 2024-05-03T20:30:35.781 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-05-03T20:30:35.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:35 smithi181 ceph-mon[29801]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-03T20:30:35.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:35 smithi181 ceph-mon[29801]: from='osd.6 [v2:172.21.15.181:6818/1561969338,v1:172.21.15.181:6819/1561969338]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-03T20:30:35.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:35 smithi181 ceph-mon[29801]: osdmap e43: 7 total, 6 up, 7 in 2024-05-03T20:30:35.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:35 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T20:30:35.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:35 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T20:30:36.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:35 smithi196 ceph-mon[32858]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-03T20:30:36.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:35 smithi196 ceph-mon[32858]: from='osd.6 [v2:172.21.15.181:6818/1561969338,v1:172.21.15.181:6819/1561969338]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-03T20:30:36.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:35 smithi196 ceph-mon[32858]: osdmap e43: 7 total, 6 up, 7 in 2024-05-03T20:30:36.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:35 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T20:30:36.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:35 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T20:30:36.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:35 smithi204 ceph-mon[33595]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-03T20:30:36.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:35 smithi204 ceph-mon[33595]: from='osd.6 [v2:172.21.15.181:6818/1561969338,v1:172.21.15.181:6819/1561969338]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-03T20:30:36.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:35 smithi204 ceph-mon[33595]: osdmap e43: 7 total, 6 up, 7 in 2024-05-03T20:30:36.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:35 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T20:30:36.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:35 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T20:30:36.379 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_2 2024-05-03T20:30:36.632 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T20:30:36.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:36 smithi181 ceph-mon[29801]: purged_snaps scrub starts 2024-05-03T20:30:36.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:36 smithi181 ceph-mon[29801]: purged_snaps scrub ok 2024-05-03T20:30:36.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T20:30:36.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:36 smithi181 ceph-mon[29801]: osd.6 [v2:172.21.15.181:6818/1561969338,v1:172.21.15.181:6819/1561969338] boot 2024-05-03T20:30:36.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:36 smithi181 ceph-mon[29801]: osdmap e44: 7 total, 7 up, 7 in 2024-05-03T20:30:36.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T20:30:36.927 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:36 smithi196 ceph-mon[32858]: purged_snaps scrub starts 2024-05-03T20:30:36.927 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:36 smithi196 ceph-mon[32858]: purged_snaps scrub ok 2024-05-03T20:30:36.927 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T20:30:36.927 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:36 smithi196 ceph-mon[32858]: osd.6 [v2:172.21.15.181:6818/1561969338,v1:172.21.15.181:6819/1561969338] boot 2024-05-03T20:30:36.927 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:36 smithi196 ceph-mon[32858]: osdmap e44: 7 total, 7 up, 7 in 2024-05-03T20:30:36.927 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T20:30:37.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:36 smithi204 ceph-mon[33595]: purged_snaps scrub starts 2024-05-03T20:30:37.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:36 smithi204 ceph-mon[33595]: purged_snaps scrub ok 2024-05-03T20:30:37.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T20:30:37.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:36 smithi204 ceph-mon[33595]: osd.6 [v2:172.21.15.181:6818/1561969338,v1:172.21.15.181:6819/1561969338] boot 2024-05-03T20:30:37.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:36 smithi204 ceph-mon[33595]: osdmap e44: 7 total, 7 up, 7 in 2024-05-03T20:30:37.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T20:30:37.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:37 smithi196 ceph-mon[32858]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-03T20:30:37.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:37 smithi196 ceph-mon[32858]: osdmap e45: 7 total, 7 up, 7 in 2024-05-03T20:30:37.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:37 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:37.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:37 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:37.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:37 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:30:37.934 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:37 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:30:37.934 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:37 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:30:37.934 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:37 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:37.934 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:37 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:37.934 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:37 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:30:37.934 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:37 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:37.934 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:37 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:30:37.934 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:37 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:37.934 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:37 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:30:37.934 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:37 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:37.955 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.953+0000 7f2db8a8a700 1 -- 172.21.15.196:0/2818390562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2db41035f0 msgr2=0x7f2db4103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:30:37.955 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.953+0000 7f2db8a8a700 1 --2- 172.21.15.196:0/2818390562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2db41035f0 0x7f2db4103a50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2da40099c0 tx=0x7f2da4009cd0 comp rx=0 tx=0).stop 2024-05-03T20:30:37.955 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.953+0000 7f2db8a8a700 1 -- 172.21.15.196:0/2818390562 shutdown_connections 2024-05-03T20:30:37.955 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.954+0000 7f2db8a8a700 1 --2- 172.21.15.196:0/2818390562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2db4103f90 0x7f2db410a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:30:37.955 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.954+0000 7f2db8a8a700 1 --2- 172.21.15.196:0/2818390562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2db41023f0 0x7f2db4102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:30:37.955 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.954+0000 7f2db8a8a700 1 --2- 172.21.15.196:0/2818390562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2db41035f0 0x7f2db4103a50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:30:37.955 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.954+0000 7f2db8a8a700 1 -- 172.21.15.196:0/2818390562 >> 172.21.15.196:0/2818390562 conn(0x7f2db40fd990 msgr2=0x7f2db40ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:30:37.955 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.954+0000 7f2db8a8a700 1 -- 172.21.15.196:0/2818390562 shutdown_connections 2024-05-03T20:30:37.955 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.954+0000 7f2db8a8a700 1 -- 172.21.15.196:0/2818390562 wait complete. 2024-05-03T20:30:37.955 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.954+0000 7f2db8a8a700 1 Processor -- start 2024-05-03T20:30:37.955 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.954+0000 7f2db8a8a700 1 -- start start 2024-05-03T20:30:37.955 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.955+0000 7f2db8a8a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2db41023f0 0x7f2db406c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:30:37.955 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.955+0000 7f2db8a8a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2db41035f0 0x7f2db419e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:30:37.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.955+0000 7f2db8a8a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2db4103f90 0x7f2db41a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:30:37.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.955+0000 7f2db8a8a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2db410df00 con 0x7f2db41023f0 2024-05-03T20:30:37.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.955+0000 7f2db8a8a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2db410dd80 con 0x7f2db41035f0 2024-05-03T20:30:37.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.955+0000 7f2db8a8a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2db410e080 con 0x7f2db4103f90 2024-05-03T20:30:37.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.955+0000 7f2db1d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2db41035f0 0x7f2db419e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:30:37.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.955+0000 7f2db1d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2db41035f0 0x7f2db419e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:60060/0 (socket says 172.21.15.196:60060) 2024-05-03T20:30:37.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.955+0000 7f2db1d9b700 1 -- 172.21.15.196:0/2395298006 learned_addr learned my addr 172.21.15.196:0/2395298006 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-05-03T20:30:37.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.955+0000 7f2db259c700 1 --2- 172.21.15.196:0/2395298006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2db41023f0 0x7f2db406c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:30:37.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.955+0000 7f2db2d9d700 1 --2- 172.21.15.196:0/2395298006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2db4103f90 0x7f2db41a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:30:37.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.955+0000 7f2db1d9b700 1 -- 172.21.15.196:0/2395298006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2db4103f90 msgr2=0x7f2db41a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:30:37.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.955+0000 7f2db1d9b700 1 --2- 172.21.15.196:0/2395298006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2db4103f90 0x7f2db41a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:30:37.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.955+0000 7f2db1d9b700 1 -- 172.21.15.196:0/2395298006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2db41023f0 msgr2=0x7f2db406c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:30:37.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.955+0000 7f2db1d9b700 1 --2- 172.21.15.196:0/2395298006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2db41023f0 0x7f2db406c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:30:37.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.955+0000 7f2db1d9b700 1 -- 172.21.15.196:0/2395298006 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db41a1380 con 0x7f2db41035f0 2024-05-03T20:30:37.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.955+0000 7f2db1d9b700 1 --2- 172.21.15.196:0/2395298006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2db41035f0 0x7f2db419e470 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2da4005910 tx=0x7f2da400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:30:37.956 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.955+0000 7f2da37fe700 1 -- 172.21.15.196:0/2395298006 <== mon.2 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da4027070 con 0x7f2db41035f0 2024-05-03T20:30:37.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.955+0000 7f2db8a8a700 1 -- 172.21.15.196:0/2395298006 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db41a1610 con 0x7f2db41035f0 2024-05-03T20:30:37.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.955+0000 7f2da37fe700 1 -- 172.21.15.196:0/2395298006 <== mon.2 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2da400fe40 con 0x7f2db41035f0 2024-05-03T20:30:37.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.955+0000 7f2da37fe700 1 -- 172.21.15.196:0/2395298006 <== mon.2 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da40173f0 con 0x7f2db41035f0 2024-05-03T20:30:37.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.955+0000 7f2db8a8a700 1 -- 172.21.15.196:0/2395298006 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db41a1ae0 con 0x7f2db41035f0 2024-05-03T20:30:37.958 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.956+0000 7f2db8a8a700 1 -- 172.21.15.196:0/2395298006 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2db406a190 con 0x7f2db41035f0 2024-05-03T20:30:37.958 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.957+0000 7f2da37fe700 1 -- 172.21.15.196:0/2395298006 <== mon.2 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f2da402a070 con 0x7f2db41035f0 2024-05-03T20:30:37.958 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.958+0000 7f2da37fe700 1 --2- 172.21.15.196:0/2395298006 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f2d9c0b2880 0x7f2d9c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:30:37.958 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.958+0000 7f2da37fe700 1 -- 172.21.15.196:0/2395298006 <== mon.2 v2:172.21.15.196:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 4965+0+0 (secure 0 0 0) 0x7f2da40e2140 con 0x7f2db41035f0 2024-05-03T20:30:37.958 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.958+0000 7f2db259c700 1 --2- 172.21.15.196:0/2395298006 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f2d9c0b2880 0x7f2d9c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:30:37.958 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.959+0000 7f2db259c700 1 --2- 172.21.15.196:0/2395298006 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f2d9c0b2880 0x7f2d9c0b4d40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2db4103450 tx=0x7f2da8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:30:37.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:37.963+0000 7f2da37fe700 1 -- 172.21.15.196:0/2395298006 <== mon.2 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2da4013070 con 0x7f2db41035f0 2024-05-03T20:30:37.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:37 smithi181 ceph-mon[29801]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-03T20:30:37.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:37 smithi181 ceph-mon[29801]: osdmap e45: 7 total, 7 up, 7 in 2024-05-03T20:30:37.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:37 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:37.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:37 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:37.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:37 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:30:37.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:37 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:30:37.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:37 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:30:37.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:37 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:37.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:37 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:37.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:37 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:30:37.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:37 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:37.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:37 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:30:37.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:37 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:37.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:37 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:30:37.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:37 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:38.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:37 smithi204 ceph-mon[33595]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-03T20:30:38.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:37 smithi204 ceph-mon[33595]: osdmap e45: 7 total, 7 up, 7 in 2024-05-03T20:30:38.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:37 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:38.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:37 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:38.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:37 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:30:38.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:37 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:30:38.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:37 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:30:38.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:37 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:38.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:37 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:38.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:37 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:30:38.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:37 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:38.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:37 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:30:38.115 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:37 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:38.115 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:37 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:30:38.115 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:37 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:38.176 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:38.177+0000 7f2db8a8a700 1 -- 172.21.15.196:0/2395298006 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2db40638e0 con 0x7f2d9c0b2880 2024-05-03T20:30:38.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:38 smithi181 ceph-mon[29801]: Detected new or changed devices on smithi181 2024-05-03T20:30:38.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:38 smithi181 ceph-mon[29801]: Adjusting osd_memory_target on smithi181 to 5696M 2024-05-03T20:30:38.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:38 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:30:38.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:38 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:30:38.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:38 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:38.985 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:38 smithi196 ceph-mon[32858]: Detected new or changed devices on smithi181 2024-05-03T20:30:38.985 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:38 smithi196 ceph-mon[32858]: Adjusting osd_memory_target on smithi181 to 5696M 2024-05-03T20:30:38.985 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:38 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:30:38.985 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:38 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:30:38.985 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:38 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:39.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:38 smithi204 ceph-mon[33595]: Detected new or changed devices on smithi181 2024-05-03T20:30:39.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:38 smithi204 ceph-mon[33595]: Adjusting osd_memory_target on smithi181 to 5696M 2024-05-03T20:30:39.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:38 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:30:39.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:38 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:30:39.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:38 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:39.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:39 smithi181 ceph-mon[29801]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:39.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:39 smithi181 ceph-mon[29801]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:30:39.983 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:39 smithi196 ceph-mon[32858]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:39.983 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:39 smithi196 ceph-mon[32858]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:30:40.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:39 smithi204 ceph-mon[33595]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:40.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:39 smithi204 ceph-mon[33595]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:30:40.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:40 smithi181 ceph-mon[29801]: from='client.? 172.21.15.196:0/3353014488' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "016c145c-ad67-42ac-9775-ca2840790abc"}]: dispatch 2024-05-03T20:30:40.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:40 smithi181 ceph-mon[29801]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "016c145c-ad67-42ac-9775-ca2840790abc"}]: dispatch 2024-05-03T20:30:40.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:40 smithi181 ceph-mon[29801]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "016c145c-ad67-42ac-9775-ca2840790abc"}]': finished 2024-05-03T20:30:40.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:40 smithi181 ceph-mon[29801]: osdmap e46: 8 total, 7 up, 8 in 2024-05-03T20:30:40.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:40 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T20:30:40.995 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:40 smithi196 ceph-mon[32858]: from='client.? 172.21.15.196:0/3353014488' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "016c145c-ad67-42ac-9775-ca2840790abc"}]: dispatch 2024-05-03T20:30:40.995 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:40 smithi196 ceph-mon[32858]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "016c145c-ad67-42ac-9775-ca2840790abc"}]: dispatch 2024-05-03T20:30:40.995 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:40 smithi196 ceph-mon[32858]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "016c145c-ad67-42ac-9775-ca2840790abc"}]': finished 2024-05-03T20:30:40.995 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:40 smithi196 ceph-mon[32858]: osdmap e46: 8 total, 7 up, 8 in 2024-05-03T20:30:40.996 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:40 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T20:30:41.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:40 smithi204 ceph-mon[33595]: from='client.? 172.21.15.196:0/3353014488' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "016c145c-ad67-42ac-9775-ca2840790abc"}]: dispatch 2024-05-03T20:30:41.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:40 smithi204 ceph-mon[33595]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "016c145c-ad67-42ac-9775-ca2840790abc"}]: dispatch 2024-05-03T20:30:41.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:40 smithi204 ceph-mon[33595]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "016c145c-ad67-42ac-9775-ca2840790abc"}]': finished 2024-05-03T20:30:41.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:40 smithi204 ceph-mon[33595]: osdmap e46: 8 total, 7 up, 8 in 2024-05-03T20:30:41.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:40 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T20:30:41.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:41 smithi181 ceph-mon[29801]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:41.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:41 smithi181 ceph-mon[29801]: from='client.? 172.21.15.196:0/971108301' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:30:42.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:41 smithi196 ceph-mon[32858]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:42.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:41 smithi196 ceph-mon[32858]: from='client.? 172.21.15.196:0/971108301' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:30:42.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:41 smithi204 ceph-mon[33595]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:42.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:41 smithi204 ceph-mon[33595]: from='client.? 172.21.15.196:0/971108301' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:30:43.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:43 smithi181 ceph-mon[29801]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:44.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:43 smithi196 ceph-mon[32858]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:44.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:43 smithi204 ceph-mon[33595]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:45.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:45 smithi181 ceph-mon[29801]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:45.974 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:45 smithi196 ceph-mon[32858]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:46.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:45 smithi204 ceph-mon[33595]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:47.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:47 smithi181 ceph-mon[29801]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:48.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:47 smithi196 ceph-mon[32858]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:48.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:47 smithi204 ceph-mon[33595]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:48.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-03T20:30:48.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:49.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-03T20:30:49.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:49.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-03T20:30:49.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:49.793 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:49 smithi196 ceph-mon[32858]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:49.793 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:49 smithi196 ceph-mon[32858]: Deploying daemon osd.7 on smithi196 2024-05-03T20:30:50.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:49 smithi204 ceph-mon[33595]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:50.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:49 smithi204 ceph-mon[33595]: Deploying daemon osd.7 on smithi196 2024-05-03T20:30:50.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:49 smithi181 ceph-mon[29801]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:50.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:49 smithi181 ceph-mon[29801]: Deploying daemon osd.7 on smithi196 2024-05-03T20:30:51.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:51 smithi196 ceph-mon[32858]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:52.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:51 smithi204 ceph-mon[33595]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:52.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:51 smithi181 ceph-mon[29801]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:53.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:53 smithi196 ceph-mon[32858]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:53.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:53 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:53.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:53 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:53.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:53 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:30:54.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:53 smithi204 ceph-mon[33595]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:54.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:53 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:54.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:53 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:54.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:53 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:30:54.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:53 smithi181 ceph-mon[29801]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:54.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:53 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:54.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:53 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:54.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:53 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:30:55.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:55 smithi196 ceph-mon[32858]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:56.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:55 smithi204 ceph-mon[33595]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:56.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:55 smithi181 ceph-mon[29801]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:56.539 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:56.540+0000 7f2da37fe700 1 -- 172.21.15.196:0/2395298006 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2db40638e0 con 0x7f2d9c0b2880 2024-05-03T20:30:56.539 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 7 on host 'smithi196' 2024-05-03T20:30:56.541 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:56.542+0000 7f2db8a8a700 1 -- 172.21.15.196:0/2395298006 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f2d9c0b2880 msgr2=0x7f2d9c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:30:56.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:56.542+0000 7f2db8a8a700 1 --2- 172.21.15.196:0/2395298006 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f2d9c0b2880 0x7f2d9c0b4d40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2db4103450 tx=0x7f2da8009190 comp rx=0 tx=0).stop 2024-05-03T20:30:56.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:56.542+0000 7f2db8a8a700 1 -- 172.21.15.196:0/2395298006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2db41035f0 msgr2=0x7f2db419e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:30:56.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:56.542+0000 7f2db8a8a700 1 --2- 172.21.15.196:0/2395298006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2db41035f0 0x7f2db419e470 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2da4005910 tx=0x7f2da400fa00 comp rx=0 tx=0).stop 2024-05-03T20:30:56.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:56.542+0000 7f2db8a8a700 1 -- 172.21.15.196:0/2395298006 shutdown_connections 2024-05-03T20:30:56.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:56.542+0000 7f2db8a8a700 1 --2- 172.21.15.196:0/2395298006 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f2d9c0b2880 0x7f2d9c0b4d40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:30:56.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:56.542+0000 7f2db8a8a700 1 --2- 172.21.15.196:0/2395298006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2db4103f90 0x7f2db41a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:30:56.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:56.542+0000 7f2db8a8a700 1 --2- 172.21.15.196:0/2395298006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2db41023f0 0x7f2db406c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:30:56.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:56.542+0000 7f2db8a8a700 1 --2- 172.21.15.196:0/2395298006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2db41035f0 0x7f2db419e470 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:30:56.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:56.542+0000 7f2db8a8a700 1 -- 172.21.15.196:0/2395298006 >> 172.21.15.196:0/2395298006 conn(0x7f2db40fd990 msgr2=0x7f2db4106a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:30:56.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:56.542+0000 7f2db8a8a700 1 -- 172.21.15.196:0/2395298006 shutdown_connections 2024-05-03T20:30:56.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:30:56.542+0000 7f2db8a8a700 1 -- 172.21.15.196:0/2395298006 wait complete. 2024-05-03T20:30:57.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:56 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:57.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:56 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:57.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:56 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:57.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:56 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:30:57.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:56 smithi196 ceph-mon[32858]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:57.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:56 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:57.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:56 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:57.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:56 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:57.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:56 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:30:57.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:56 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:57.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:56 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:57.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:56 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:57.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:56 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:30:57.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:56 smithi204 ceph-mon[33595]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:57.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:56 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:57.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:56 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:57.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:56 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:57.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:56 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:30:57.115 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:56 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:57.115 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:56 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:57.115 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:56 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:30:57.115 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:56 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:30:57.115 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:56 smithi181 ceph-mon[29801]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:57.115 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:56 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:57.115 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:56 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:57.115 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:56 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:30:57.116 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:56 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:30:57.176 DEBUG:teuthology.orchestra.run.smithi196:osd.7> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.7.service 2024-05-03T20:30:57.178 INFO:tasks.cephadm:Deploying osd.8 on smithi204 with /dev/vg_nvme/lv_2... 2024-05-03T20:30:57.178 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-03T20:30:57.426 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T20:30:58.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:57 smithi196 ceph-mon[32858]: from='osd.7 [v2:172.21.15.196:6816/824890163,v1:172.21.15.196:6817/824890163]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-03T20:30:58.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:57 smithi196 ceph-mon[32858]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-03T20:30:58.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:57 smithi204 ceph-mon[33595]: from='osd.7 [v2:172.21.15.196:6816/824890163,v1:172.21.15.196:6817/824890163]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-03T20:30:58.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:57 smithi204 ceph-mon[33595]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-03T20:30:58.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:57 smithi181 ceph-mon[29801]: from='osd.7 [v2:172.21.15.196:6816/824890163,v1:172.21.15.196:6817/824890163]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-03T20:30:58.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:57 smithi181 ceph-mon[29801]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-03T20:30:59.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:58 smithi196 ceph-mon[32858]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:59.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:58 smithi196 ceph-mon[32858]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-03T20:30:59.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:58 smithi196 ceph-mon[32858]: osdmap e47: 8 total, 7 up, 8 in 2024-05-03T20:30:59.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:58 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T20:30:59.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:58 smithi196 ceph-mon[32858]: from='osd.7 [v2:172.21.15.196:6816/824890163,v1:172.21.15.196:6817/824890163]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-03T20:30:59.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:58 smithi196 ceph-mon[32858]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-03T20:30:59.109 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:30:58 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:30:58.838+0000 7f14010bb700 -1 osd.7 0 waiting for initial osdmap 2024-05-03T20:30:59.109 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:30:58 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:30:58.846+0000 7f13fceae700 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T20:30:59.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:58 smithi204 ceph-mon[33595]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:59.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:58 smithi204 ceph-mon[33595]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-03T20:30:59.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:58 smithi204 ceph-mon[33595]: osdmap e47: 8 total, 7 up, 8 in 2024-05-03T20:30:59.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:58 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T20:30:59.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:58 smithi204 ceph-mon[33595]: from='osd.7 [v2:172.21.15.196:6816/824890163,v1:172.21.15.196:6817/824890163]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-03T20:30:59.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:58 smithi204 ceph-mon[33595]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-03T20:30:59.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:58 smithi181 ceph-mon[29801]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:30:59.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:58 smithi181 ceph-mon[29801]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-03T20:30:59.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:58 smithi181 ceph-mon[29801]: osdmap e47: 8 total, 7 up, 8 in 2024-05-03T20:30:59.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:58 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T20:30:59.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:58 smithi181 ceph-mon[29801]: from='osd.7 [v2:172.21.15.196:6816/824890163,v1:172.21.15.196:6817/824890163]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-03T20:30:59.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:58 smithi181 ceph-mon[29801]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-03T20:30:59.374 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-03T20:30:59.374 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-03T20:30:59.374 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-05-03T20:30:59.374 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-05-03T20:30:59.374 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131411 s, 798 MB/s 2024-05-03T20:30:59.374 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-05-03T20:30:59.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:59 smithi196 ceph-mon[32858]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-03T20:30:59.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:59 smithi196 ceph-mon[32858]: osdmap e48: 8 total, 7 up, 8 in 2024-05-03T20:30:59.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:59 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T20:30:59.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:59 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T20:30:59.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:59 smithi196 ceph-mon[32858]: osd.7 [v2:172.21.15.196:6816/824890163,v1:172.21.15.196:6817/824890163] boot 2024-05-03T20:30:59.923 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:59 smithi196 ceph-mon[32858]: osdmap e49: 8 total, 8 up, 8 in 2024-05-03T20:30:59.923 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:30:59 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T20:30:59.955 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:59 smithi204 ceph-mon[33595]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-03T20:30:59.955 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:59 smithi204 ceph-mon[33595]: osdmap e48: 8 total, 7 up, 8 in 2024-05-03T20:30:59.955 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:59 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T20:30:59.955 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:59 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T20:30:59.955 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:59 smithi204 ceph-mon[33595]: osd.7 [v2:172.21.15.196:6816/824890163,v1:172.21.15.196:6817/824890163] boot 2024-05-03T20:30:59.955 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:59 smithi204 ceph-mon[33595]: osdmap e49: 8 total, 8 up, 8 in 2024-05-03T20:30:59.956 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:30:59 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T20:30:59.957 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_2 2024-05-03T20:31:00.198 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T20:31:00.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:59 smithi181 ceph-mon[29801]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-03T20:31:00.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:59 smithi181 ceph-mon[29801]: osdmap e48: 8 total, 7 up, 8 in 2024-05-03T20:31:00.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:59 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T20:31:00.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:59 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T20:31:00.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:59 smithi181 ceph-mon[29801]: osd.7 [v2:172.21.15.196:6816/824890163,v1:172.21.15.196:6817/824890163] boot 2024-05-03T20:31:00.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:59 smithi181 ceph-mon[29801]: osdmap e49: 8 total, 8 up, 8 in 2024-05-03T20:31:00.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:30:59 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T20:31:00.993 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:00 smithi204 ceph-mon[33595]: purged_snaps scrub starts 2024-05-03T20:31:00.994 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:00 smithi204 ceph-mon[33595]: purged_snaps scrub ok 2024-05-03T20:31:00.994 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:00 smithi204 ceph-mon[33595]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:31:00.994 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:00 smithi204 ceph-mon[33595]: Detected new or changed devices on smithi196 2024-05-03T20:31:00.994 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:00 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:00.994 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:00 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:00.994 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:00 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:00.994 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:00 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:00.994 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:00 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:00.994 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:00 smithi204 ceph-mon[33595]: Adjusting osd_memory_target on smithi196 to 5696M 2024-05-03T20:31:00.994 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:00 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:00.994 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:00 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:00.995 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:00 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:31:00.995 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:00 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:00.995 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:00 smithi204 ceph-mon[33595]: osdmap e50: 8 total, 8 up, 8 in 2024-05-03T20:31:01.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:00 smithi196 ceph-mon[32858]: purged_snaps scrub starts 2024-05-03T20:31:01.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:00 smithi196 ceph-mon[32858]: purged_snaps scrub ok 2024-05-03T20:31:01.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:00 smithi196 ceph-mon[32858]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:31:01.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:00 smithi196 ceph-mon[32858]: Detected new or changed devices on smithi196 2024-05-03T20:31:01.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:00 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:01.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:00 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:01.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:00 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:01.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:00 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:01.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:00 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:01.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:00 smithi196 ceph-mon[32858]: Adjusting osd_memory_target on smithi196 to 5696M 2024-05-03T20:31:01.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:00 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:01.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:00 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:01.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:00 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:31:01.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:00 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:01.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:00 smithi196 ceph-mon[32858]: osdmap e50: 8 total, 8 up, 8 in 2024-05-03T20:31:01.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:00 smithi181 ceph-mon[29801]: purged_snaps scrub starts 2024-05-03T20:31:01.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:00 smithi181 ceph-mon[29801]: purged_snaps scrub ok 2024-05-03T20:31:01.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:00 smithi181 ceph-mon[29801]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-03T20:31:01.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:00 smithi181 ceph-mon[29801]: Detected new or changed devices on smithi196 2024-05-03T20:31:01.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:00 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:01.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:00 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:01.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:00 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:01.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:00 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:01.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:00 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:01.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:00 smithi181 ceph-mon[29801]: Adjusting osd_memory_target on smithi196 to 5696M 2024-05-03T20:31:01.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:00 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:01.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:00 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:01.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:00 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:31:01.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:00 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:01.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:00 smithi181 ceph-mon[29801]: osdmap e50: 8 total, 8 up, 8 in 2024-05-03T20:31:01.770 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.768+0000 7fc1a1e2d700 1 -- 172.21.15.204:0/537893617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc19c1017b0 msgr2=0x7fc19c101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:31:01.770 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.768+0000 7fc1a1e2d700 1 --2- 172.21.15.204:0/537893617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc19c1017b0 0x7fc19c101bd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009990 tx=0x7fc18c009ca0 comp rx=0 tx=0).stop 2024-05-03T20:31:01.770 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.768+0000 7fc1a1e2d700 1 -- 172.21.15.204:0/537893617 shutdown_connections 2024-05-03T20:31:01.770 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.768+0000 7fc1a1e2d700 1 --2- 172.21.15.204:0/537893617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc19c105130 0x7fc19c10b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:31:01.770 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.768+0000 7fc1a1e2d700 1 --2- 172.21.15.204:0/537893617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc19c1017b0 0x7fc19c101bd0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:31:01.770 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.768+0000 7fc1a1e2d700 1 --2- 172.21.15.204:0/537893617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc19c104790 0x7fc19c104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:31:01.771 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.768+0000 7fc1a1e2d700 1 -- 172.21.15.204:0/537893617 >> 172.21.15.204:0/537893617 conn(0x7fc19c0fd130 msgr2=0x7fc19c0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:31:01.771 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.768+0000 7fc1a1e2d700 1 -- 172.21.15.204:0/537893617 shutdown_connections 2024-05-03T20:31:01.771 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.768+0000 7fc1a1e2d700 1 -- 172.21.15.204:0/537893617 wait complete. 2024-05-03T20:31:01.771 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.769+0000 7fc1a1e2d700 1 Processor -- start 2024-05-03T20:31:01.771 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.769+0000 7fc1a1e2d700 1 -- start start 2024-05-03T20:31:01.771 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.769+0000 7fc1a1e2d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc19c1017b0 0x7fc19c199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:31:01.771 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.769+0000 7fc1a1e2d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc19c104790 0x7fc19c199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:31:01.771 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.769+0000 7fc1a1e2d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc19c105130 0x7fc19c1a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:31:01.771 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.769+0000 7fc1a1e2d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c10f000 con 0x7fc19c105130 2024-05-03T20:31:01.771 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.769+0000 7fc1a1e2d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c10ee80 con 0x7fc19c1017b0 2024-05-03T20:31:01.771 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.769+0000 7fc1a1e2d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c10f180 con 0x7fc19c104790 2024-05-03T20:31:01.772 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.769+0000 7fc192ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc19c104790 0x7fc19c199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:31:01.772 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.769+0000 7fc192ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc19c104790 0x7fc19c199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:36848/0 (socket says 172.21.15.204:36848) 2024-05-03T20:31:01.772 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.769+0000 7fc192ffd700 1 -- 172.21.15.204:0/2103436406 learned_addr learned my addr 172.21.15.204:0/2103436406 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-03T20:31:01.772 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.769+0000 7fc19b7fe700 1 --2- 172.21.15.204:0/2103436406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc19c1017b0 0x7fc19c199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:31:01.772 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.769+0000 7fc19bfff700 1 --2- 172.21.15.204:0/2103436406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc19c105130 0x7fc19c1a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:31:01.772 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.770+0000 7fc192ffd700 1 -- 172.21.15.204:0/2103436406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc19c1017b0 msgr2=0x7fc19c199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:31:01.772 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.770+0000 7fc192ffd700 1 --2- 172.21.15.204:0/2103436406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc19c1017b0 0x7fc19c199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:31:01.772 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.770+0000 7fc192ffd700 1 -- 172.21.15.204:0/2103436406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc19c105130 msgr2=0x7fc19c1a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:31:01.772 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.770+0000 7fc192ffd700 1 --2- 172.21.15.204:0/2103436406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc19c105130 0x7fc19c1a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:31:01.772 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.770+0000 7fc192ffd700 1 -- 172.21.15.204:0/2103436406 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19c1a12e0 con 0x7fc19c104790 2024-05-03T20:31:01.772 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.770+0000 7fc192ffd700 1 --2- 172.21.15.204:0/2103436406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc19c104790 0x7fc19c199bc0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc18400b560 tx=0x7fc18400b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:31:01.773 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.770+0000 7fc1997fa700 1 -- 172.21.15.204:0/2103436406 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184015020 con 0x7fc19c104790 2024-05-03T20:31:01.773 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.770+0000 7fc1a1e2d700 1 -- 172.21.15.204:0/2103436406 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc19c1a15d0 con 0x7fc19c104790 2024-05-03T20:31:01.773 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.770+0000 7fc1997fa700 1 -- 172.21.15.204:0/2103436406 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc184011ce0 con 0x7fc19c104790 2024-05-03T20:31:01.773 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.770+0000 7fc1997fa700 1 -- 172.21.15.204:0/2103436406 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184010d40 con 0x7fc19c104790 2024-05-03T20:31:01.773 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.770+0000 7fc1a1e2d700 1 -- 172.21.15.204:0/2103436406 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc19c1a1be0 con 0x7fc19c104790 2024-05-03T20:31:01.774 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.771+0000 7fc1a1e2d700 1 -- 172.21.15.204:0/2103436406 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc19c06a190 con 0x7fc19c104790 2024-05-03T20:31:01.774 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.772+0000 7fc1997fa700 1 -- 172.21.15.204:0/2103436406 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fc184018020 con 0x7fc19c104790 2024-05-03T20:31:01.774 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.773+0000 7fc1997fa700 1 --2- 172.21.15.204:0/2103436406 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fc1880b2880 0x7fc1880b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:31:01.774 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.773+0000 7fc1997fa700 1 -- 172.21.15.204:0/2103436406 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5397+0+0 (secure 0 0 0) 0x7fc1840e0d20 con 0x7fc19c104790 2024-05-03T20:31:01.775 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.773+0000 7fc19b7fe700 1 --2- 172.21.15.204:0/2103436406 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fc1880b2880 0x7fc1880b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:31:01.775 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.773+0000 7fc19b7fe700 1 --2- 172.21.15.204:0/2103436406 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fc1880b2880 0x7fc1880b4d40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc18c0038a0 tx=0x7fc18c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:31:01.776 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.778+0000 7fc1997fa700 1 -- 172.21.15.204:0/2103436406 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc18401f020 con 0x7fc19c104790 2024-05-03T20:31:01.992 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:01.993+0000 7fc1a1e2d700 1 -- 172.21.15.204:0/2103436406 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc19c0638e0 con 0x7fc1880b2880 2024-05-03T20:31:02.687 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:02 smithi204 ceph-mon[33595]: osdmap e51: 8 total, 8 up, 8 in 2024-05-03T20:31:02.687 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:02 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:31:02.687 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:02 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:31:02.687 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:02 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:02.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:02 smithi181 ceph-mon[29801]: osdmap e51: 8 total, 8 up, 8 in 2024-05-03T20:31:02.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:02 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:31:02.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:02 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:31:02.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:02 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:02.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:02 smithi196 ceph-mon[32858]: osdmap e51: 8 total, 8 up, 8 in 2024-05-03T20:31:02.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:02 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:31:02.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:02 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:31:02.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:02 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:03.690 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:03 smithi204 ceph-mon[33595]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-03T20:31:03.690 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:03 smithi204 ceph-mon[33595]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:31:03.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:03 smithi181 ceph-mon[29801]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-03T20:31:03.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:03 smithi181 ceph-mon[29801]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:31:03.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:03 smithi196 ceph-mon[32858]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-03T20:31:03.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:03 smithi196 ceph-mon[32858]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:31:05.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:05 smithi181 ceph-mon[29801]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-03T20:31:05.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:05 smithi181 ceph-mon[29801]: from='client.? 172.21.15.204:0/2694739561' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10235638-c1f1-41d3-8e24-f8b2344bfeff"}]: dispatch 2024-05-03T20:31:05.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:05 smithi181 ceph-mon[29801]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10235638-c1f1-41d3-8e24-f8b2344bfeff"}]: dispatch 2024-05-03T20:31:05.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:05 smithi181 ceph-mon[29801]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10235638-c1f1-41d3-8e24-f8b2344bfeff"}]': finished 2024-05-03T20:31:05.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:05 smithi181 ceph-mon[29801]: osdmap e52: 9 total, 8 up, 9 in 2024-05-03T20:31:05.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:05 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T20:31:05.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:05 smithi181 ceph-mon[29801]: from='client.? 172.21.15.204:0/1279294250' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:31:05.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:05 smithi196 ceph-mon[32858]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-03T20:31:05.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:05 smithi196 ceph-mon[32858]: from='client.? 172.21.15.204:0/2694739561' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10235638-c1f1-41d3-8e24-f8b2344bfeff"}]: dispatch 2024-05-03T20:31:05.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:05 smithi196 ceph-mon[32858]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10235638-c1f1-41d3-8e24-f8b2344bfeff"}]: dispatch 2024-05-03T20:31:05.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:05 smithi196 ceph-mon[32858]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10235638-c1f1-41d3-8e24-f8b2344bfeff"}]': finished 2024-05-03T20:31:05.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:05 smithi196 ceph-mon[32858]: osdmap e52: 9 total, 8 up, 9 in 2024-05-03T20:31:05.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:05 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T20:31:05.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:05 smithi196 ceph-mon[32858]: from='client.? 172.21.15.204:0/1279294250' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:31:05.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:05 smithi204 ceph-mon[33595]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-03T20:31:05.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:05 smithi204 ceph-mon[33595]: from='client.? 172.21.15.204:0/2694739561' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10235638-c1f1-41d3-8e24-f8b2344bfeff"}]: dispatch 2024-05-03T20:31:05.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:05 smithi204 ceph-mon[33595]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10235638-c1f1-41d3-8e24-f8b2344bfeff"}]: dispatch 2024-05-03T20:31:05.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:05 smithi204 ceph-mon[33595]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10235638-c1f1-41d3-8e24-f8b2344bfeff"}]': finished 2024-05-03T20:31:05.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:05 smithi204 ceph-mon[33595]: osdmap e52: 9 total, 8 up, 9 in 2024-05-03T20:31:05.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:05 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T20:31:05.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:05 smithi204 ceph-mon[33595]: from='client.? 172.21.15.204:0/1279294250' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:31:07.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:07 smithi181 ceph-mon[29801]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T20:31:07.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:07 smithi196 ceph-mon[32858]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T20:31:07.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:07 smithi204 ceph-mon[33595]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T20:31:09.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:09 smithi181 ceph-mon[29801]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-03T20:31:09.727 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:09 smithi204 ceph-mon[33595]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-03T20:31:09.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:09 smithi196 ceph-mon[32858]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-03T20:31:11.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:11 smithi204 ceph-mon[33595]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-03T20:31:11.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:11 smithi181 ceph-mon[29801]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-03T20:31:11.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:11 smithi196 ceph-mon[32858]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-03T20:31:12.728 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-03T20:31:12.728 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:12.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-03T20:31:12.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:12.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-03T20:31:12.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:13.748 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:13 smithi204 ceph-mon[33595]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T20:31:13.748 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:13 smithi204 ceph-mon[33595]: Deploying daemon osd.8 on smithi204 2024-05-03T20:31:13.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:13 smithi196 ceph-mon[32858]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T20:31:13.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:13 smithi196 ceph-mon[32858]: Deploying daemon osd.8 on smithi204 2024-05-03T20:31:13.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:13 smithi181 ceph-mon[29801]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T20:31:13.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:13 smithi181 ceph-mon[29801]: Deploying daemon osd.8 on smithi204 2024-05-03T20:31:15.735 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:15 smithi204 ceph-mon[33595]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T20:31:15.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:15 smithi196 ceph-mon[32858]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T20:31:15.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:15 smithi181 ceph-mon[29801]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T20:31:17.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:17 smithi204 ceph-mon[33595]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-05-03T20:31:17.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:17 smithi196 ceph-mon[32858]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-05-03T20:31:17.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:17 smithi181 ceph-mon[29801]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-05-03T20:31:18.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:18 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:18.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:18 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:18.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:18 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:31:18.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:18 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:18.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:18 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:18.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:18 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:31:18.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:18 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:18.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:18 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:18.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:18 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:31:19.788 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:19 smithi204 ceph-mon[33595]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T20:31:19.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:19 smithi196 ceph-mon[32858]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T20:31:19.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:19 smithi181 ceph-mon[29801]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T20:31:20.907 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:20.908+0000 7fc1997fa700 1 -- 172.21.15.204:0/2103436406 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc19c0638e0 con 0x7fc1880b2880 2024-05-03T20:31:20.907 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 8 on host 'smithi204' 2024-05-03T20:31:20.910 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:20.910+0000 7fc1a1e2d700 1 -- 172.21.15.204:0/2103436406 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fc1880b2880 msgr2=0x7fc1880b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:31:20.910 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:20.910+0000 7fc1a1e2d700 1 --2- 172.21.15.204:0/2103436406 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fc1880b2880 0x7fc1880b4d40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc18c0038a0 tx=0x7fc18c005fb0 comp rx=0 tx=0).stop 2024-05-03T20:31:20.910 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:20.910+0000 7fc1a1e2d700 1 -- 172.21.15.204:0/2103436406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc19c104790 msgr2=0x7fc19c199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:31:20.910 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:20.910+0000 7fc1a1e2d700 1 --2- 172.21.15.204:0/2103436406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc19c104790 0x7fc19c199bc0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc18400b560 tx=0x7fc18400b870 comp rx=0 tx=0).stop 2024-05-03T20:31:20.910 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:20.910+0000 7fc1a1e2d700 1 -- 172.21.15.204:0/2103436406 shutdown_connections 2024-05-03T20:31:20.910 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:20.910+0000 7fc1a1e2d700 1 --2- 172.21.15.204:0/2103436406 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fc1880b2880 0x7fc1880b4d40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:31:20.910 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:20.910+0000 7fc1a1e2d700 1 --2- 172.21.15.204:0/2103436406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc19c105130 0x7fc19c1a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:31:20.910 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:20.910+0000 7fc1a1e2d700 1 --2- 172.21.15.204:0/2103436406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc19c1017b0 0x7fc19c199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:31:20.910 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:20.910+0000 7fc1a1e2d700 1 --2- 172.21.15.204:0/2103436406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc19c104790 0x7fc19c199bc0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:31:20.910 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:20.910+0000 7fc1a1e2d700 1 -- 172.21.15.204:0/2103436406 >> 172.21.15.204:0/2103436406 conn(0x7fc19c0fd130 msgr2=0x7fc19c107bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:31:20.910 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:20.911+0000 7fc1a1e2d700 1 -- 172.21.15.204:0/2103436406 shutdown_connections 2024-05-03T20:31:20.911 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:31:20.911+0000 7fc1a1e2d700 1 -- 172.21.15.204:0/2103436406 wait complete. 2024-05-03T20:31:21.509 DEBUG:teuthology.orchestra.run.smithi204:osd.8> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.8.service 2024-05-03T20:31:21.512 INFO:tasks.cephadm:Deploying osd.9 on smithi181 with /dev/vg_nvme/lv_1... 2024-05-03T20:31:21.512 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-03T20:31:21.764 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:31:21.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:21 smithi196 ceph-mon[32858]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T20:31:21.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:21 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:21.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:21 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:21.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:21 smithi196 ceph-mon[32858]: from='osd.8 [v2:172.21.15.204:6816/3566909552,v1:172.21.15.204:6817/3566909552]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T20:31:21.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:21 smithi196 ceph-mon[32858]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T20:31:21.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:21 smithi204 ceph-mon[33595]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T20:31:21.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:21 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:21.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:21 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:21.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:21 smithi204 ceph-mon[33595]: from='osd.8 [v2:172.21.15.204:6816/3566909552,v1:172.21.15.204:6817/3566909552]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T20:31:21.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:21 smithi204 ceph-mon[33595]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T20:31:21.887 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:21 smithi181 ceph-mon[29801]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T20:31:21.888 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:21 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:21.888 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:21 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:21.888 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:21 smithi181 ceph-mon[29801]: from='osd.8 [v2:172.21.15.204:6816/3566909552,v1:172.21.15.204:6817/3566909552]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T20:31:21.888 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:21 smithi181 ceph-mon[29801]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T20:31:22.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:22 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:22.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:22 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:22.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:22 smithi196 ceph-mon[32858]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-03T20:31:22.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:22 smithi196 ceph-mon[32858]: osdmap e53: 9 total, 8 up, 9 in 2024-05-03T20:31:22.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:22 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T20:31:22.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:22 smithi196 ceph-mon[32858]: from='osd.8 [v2:172.21.15.204:6816/3566909552,v1:172.21.15.204:6817/3566909552]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-03T20:31:22.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:22 smithi196 ceph-mon[32858]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-03T20:31:22.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:22 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:22.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:22 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:22.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:22 smithi204 ceph-mon[33595]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-03T20:31:22.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:22 smithi204 ceph-mon[33595]: osdmap e53: 9 total, 8 up, 9 in 2024-05-03T20:31:22.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:22 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T20:31:22.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:22 smithi204 ceph-mon[33595]: from='osd.8 [v2:172.21.15.204:6816/3566909552,v1:172.21.15.204:6817/3566909552]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-03T20:31:22.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:22 smithi204 ceph-mon[33595]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-03T20:31:22.961 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:22 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:22.961 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:22 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:22.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:22 smithi181 ceph-mon[29801]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-03T20:31:22.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:22 smithi181 ceph-mon[29801]: osdmap e53: 9 total, 8 up, 9 in 2024-05-03T20:31:22.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:22 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T20:31:22.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:22 smithi181 ceph-mon[29801]: from='osd.8 [v2:172.21.15.204:6816/3566909552,v1:172.21.15.204:6817/3566909552]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-03T20:31:22.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:22 smithi181 ceph-mon[29801]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-03T20:31:23.802 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-03T20:31:23.802 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-03T20:31:23.802 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2024-05-03T20:31:23.802 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2024-05-03T20:31:23.802 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114093 s, 919 MB/s 2024-05-03T20:31:23.802 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2024-05-03T20:31:23.806 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:23 smithi204 ceph-mon[33595]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T20:31:23.806 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:23 smithi204 ceph-mon[33595]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-03T20:31:23.806 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:23 smithi204 ceph-mon[33595]: osdmap e54: 9 total, 8 up, 9 in 2024-05-03T20:31:23.806 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:23 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T20:31:23.806 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:23 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T20:31:23.828 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:23 smithi181 ceph-mon[29801]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T20:31:23.828 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:23 smithi181 ceph-mon[29801]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-03T20:31:23.828 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:23 smithi181 ceph-mon[29801]: osdmap e54: 9 total, 8 up, 9 in 2024-05-03T20:31:23.828 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:23 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T20:31:23.828 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:23 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T20:31:23.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:23 smithi196 ceph-mon[32858]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T20:31:23.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:23 smithi196 ceph-mon[32858]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-03T20:31:23.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:23 smithi196 ceph-mon[32858]: osdmap e54: 9 total, 8 up, 9 in 2024-05-03T20:31:23.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:23 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T20:31:23.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:23 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T20:31:24.114 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:31:23 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:31:23.802+0000 7f6c4d782700 -1 osd.8 0 waiting for initial osdmap 2024-05-03T20:31:24.114 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:31:23 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:31:23.939+0000 7f6c44d6c700 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T20:31:24.589 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_1 2024-05-03T20:31:24.843 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:31:24.876 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:24 smithi181 ceph-mon[29801]: purged_snaps scrub starts 2024-05-03T20:31:24.876 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:24 smithi181 ceph-mon[29801]: purged_snaps scrub ok 2024-05-03T20:31:24.876 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:24 smithi181 ceph-mon[29801]: from='osd.8 ' entity='osd.8' 2024-05-03T20:31:24.876 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:24 smithi181 ceph-mon[29801]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T20:31:24.877 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:24 smithi181 ceph-mon[29801]: osdmap e55: 9 total, 8 up, 9 in 2024-05-03T20:31:24.877 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:24 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T20:31:24.877 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:24 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T20:31:24.947 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:24 smithi204 ceph-mon[33595]: purged_snaps scrub starts 2024-05-03T20:31:24.947 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:24 smithi204 ceph-mon[33595]: purged_snaps scrub ok 2024-05-03T20:31:24.947 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:24 smithi204 ceph-mon[33595]: from='osd.8 ' entity='osd.8' 2024-05-03T20:31:24.947 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:24 smithi204 ceph-mon[33595]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T20:31:24.947 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:24 smithi204 ceph-mon[33595]: osdmap e55: 9 total, 8 up, 9 in 2024-05-03T20:31:24.947 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:24 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T20:31:24.947 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:24 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T20:31:25.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:24 smithi196 ceph-mon[32858]: purged_snaps scrub starts 2024-05-03T20:31:25.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:24 smithi196 ceph-mon[32858]: purged_snaps scrub ok 2024-05-03T20:31:25.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:24 smithi196 ceph-mon[32858]: from='osd.8 ' entity='osd.8' 2024-05-03T20:31:25.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:24 smithi196 ceph-mon[32858]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T20:31:25.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:24 smithi196 ceph-mon[32858]: osdmap e55: 9 total, 8 up, 9 in 2024-05-03T20:31:25.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:24 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T20:31:25.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:24 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T20:31:26.151 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:25 smithi181 ceph-mon[29801]: osd.8 [v2:172.21.15.204:6816/3566909552,v1:172.21.15.204:6817/3566909552] boot 2024-05-03T20:31:26.151 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:25 smithi181 ceph-mon[29801]: osdmap e56: 9 total, 9 up, 9 in 2024-05-03T20:31:26.151 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T20:31:26.151 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:25 smithi181 ceph-mon[29801]: Detected new or changed devices on smithi204 2024-05-03T20:31:26.151 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:26.151 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:26.152 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:26.152 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:26.152 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:26.152 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:25 smithi181 ceph-mon[29801]: Adjusting osd_memory_target on smithi204 to 5696M 2024-05-03T20:31:26.152 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:26.152 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:26.152 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:31:26.152 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:26.152 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:31:26.152 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:26.152 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:31:26.152 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:26.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:25 smithi196 ceph-mon[32858]: osd.8 [v2:172.21.15.204:6816/3566909552,v1:172.21.15.204:6817/3566909552] boot 2024-05-03T20:31:26.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:25 smithi196 ceph-mon[32858]: osdmap e56: 9 total, 9 up, 9 in 2024-05-03T20:31:26.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T20:31:26.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:25 smithi196 ceph-mon[32858]: Detected new or changed devices on smithi204 2024-05-03T20:31:26.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:26.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:26.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:26.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:26.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:26.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:25 smithi196 ceph-mon[32858]: Adjusting osd_memory_target on smithi204 to 5696M 2024-05-03T20:31:26.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:26.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:26.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:31:26.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:26.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:31:26.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:26.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:31:26.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:26.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:25 smithi204 ceph-mon[33595]: osd.8 [v2:172.21.15.204:6816/3566909552,v1:172.21.15.204:6817/3566909552] boot 2024-05-03T20:31:26.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:25 smithi204 ceph-mon[33595]: osdmap e56: 9 total, 9 up, 9 in 2024-05-03T20:31:26.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T20:31:26.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:25 smithi204 ceph-mon[33595]: Detected new or changed devices on smithi204 2024-05-03T20:31:26.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:26.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:26.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:26.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:26.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:26.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:25 smithi204 ceph-mon[33595]: Adjusting osd_memory_target on smithi204 to 5696M 2024-05-03T20:31:26.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:26.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:26.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:31:26.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:26.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:31:26.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:26.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:31:26.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:26.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.369+0000 7f251db7a700 1 -- 172.21.15.181:0/540320300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25181034f0 msgr2=0x7f2518103910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:31:26.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.369+0000 7f251db7a700 1 --2- 172.21.15.181:0/540320300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25181034f0 0x7f2518103910 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f250c0099c0 tx=0x7f250c009cd0 comp rx=0 tx=0).stop 2024-05-03T20:31:26.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.369+0000 7f251db7a700 1 -- 172.21.15.181:0/540320300 shutdown_connections 2024-05-03T20:31:26.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.369+0000 7f251db7a700 1 --2- 172.21.15.181:0/540320300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2518105090 0x7f251810b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:31:26.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.369+0000 7f251db7a700 1 --2- 172.21.15.181:0/540320300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25181034f0 0x7f2518103910 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:31:26.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.369+0000 7f251db7a700 1 --2- 172.21.15.181:0/540320300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25181046f0 0x7f2518104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:31:26.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.369+0000 7f251db7a700 1 -- 172.21.15.181:0/540320300 >> 172.21.15.181:0/540320300 conn(0x7f25180fea90 msgr2=0x7f2518100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:31:26.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.369+0000 7f251db7a700 1 -- 172.21.15.181:0/540320300 shutdown_connections 2024-05-03T20:31:26.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.369+0000 7f251db7a700 1 -- 172.21.15.181:0/540320300 wait complete. 2024-05-03T20:31:26.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.370+0000 7f251db7a700 1 Processor -- start 2024-05-03T20:31:26.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.371+0000 7f251db7a700 1 -- start start 2024-05-03T20:31:26.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.371+0000 7f251db7a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25181034f0 0x7f2518199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:31:26.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.371+0000 7f251db7a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25181046f0 0x7f2518199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:31:26.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.371+0000 7f251db7a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2518105090 0x7f25181a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:31:26.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.371+0000 7f251db7a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f251810ef80 con 0x7f25181034f0 2024-05-03T20:31:26.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.371+0000 7f251db7a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f251810ee00 con 0x7f2518105090 2024-05-03T20:31:26.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.371+0000 7f251db7a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f251810f100 con 0x7f25181046f0 2024-05-03T20:31:26.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.371+0000 7f25177fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25181034f0 0x7f2518199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:31:26.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.371+0000 7f25177fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25181034f0 0x7f2518199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58056/0 (socket says 172.21.15.181:58056) 2024-05-03T20:31:26.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.371+0000 7f25177fe700 1 -- 172.21.15.181:0/125323857 learned_addr learned my addr 172.21.15.181:0/125323857 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:31:26.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.371+0000 7f2516ffd700 1 --2- 172.21.15.181:0/125323857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25181046f0 0x7f2518199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:31:26.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.371+0000 7f2517fff700 1 --2- 172.21.15.181:0/125323857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2518105090 0x7f25181a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:31:26.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.371+0000 7f25177fe700 1 -- 172.21.15.181:0/125323857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25181046f0 msgr2=0x7f2518199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:31:26.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.371+0000 7f25177fe700 1 --2- 172.21.15.181:0/125323857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25181046f0 0x7f2518199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:31:26.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.371+0000 7f25177fe700 1 -- 172.21.15.181:0/125323857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2518105090 msgr2=0x7f25181a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:31:26.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.371+0000 7f25177fe700 1 --2- 172.21.15.181:0/125323857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2518105090 0x7f25181a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:31:26.373 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.371+0000 7f25177fe700 1 -- 172.21.15.181:0/125323857 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25181a1290 con 0x7f25181034f0 2024-05-03T20:31:26.373 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.371+0000 7f2517fff700 1 --2- 172.21.15.181:0/125323857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2518105090 0x7f25181a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:31:26.373 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.371+0000 7f25177fe700 1 --2- 172.21.15.181:0/125323857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25181034f0 0x7f2518199630 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f250c009f50 tx=0x7f250c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:31:26.373 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.371+0000 7f2514ff9700 1 -- 172.21.15.181:0/125323857 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c027070 con 0x7f25181034f0 2024-05-03T20:31:26.373 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.371+0000 7f2514ff9700 1 -- 172.21.15.181:0/125323857 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f250c0040a0 con 0x7f25181034f0 2024-05-03T20:31:26.373 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.371+0000 7f251db7a700 1 -- 172.21.15.181:0/125323857 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25181a1520 con 0x7f25181034f0 2024-05-03T20:31:26.373 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.371+0000 7f2514ff9700 1 -- 172.21.15.181:0/125323857 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c022da0 con 0x7f25181034f0 2024-05-03T20:31:26.373 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.372+0000 7f251db7a700 1 -- 172.21.15.181:0/125323857 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25181a1a00 con 0x7f25181034f0 2024-05-03T20:31:26.373 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.372+0000 7f251db7a700 1 -- 172.21.15.181:0/125323857 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f251806a190 con 0x7f25181034f0 2024-05-03T20:31:26.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.374+0000 7f2514ff9700 1 -- 172.21.15.181:0/125323857 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f250c02a070 con 0x7f25181034f0 2024-05-03T20:31:26.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.375+0000 7f2514ff9700 1 --2- 172.21.15.181:0/125323857 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f25000b2880 0x7f25000b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:31:26.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.375+0000 7f2516ffd700 1 --2- 172.21.15.181:0/125323857 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f25000b2880 0x7f25000b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:31:26.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.375+0000 7f2514ff9700 1 -- 172.21.15.181:0/125323857 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5829+0+0 (secure 0 0 0) 0x7f250c0e29e0 con 0x7f25181034f0 2024-05-03T20:31:26.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.375+0000 7f2516ffd700 1 --2- 172.21.15.181:0/125323857 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f25000b2880 0x7f25000b4d40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2508005950 tx=0x7f250800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:31:26.378 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.379+0000 7f2514ff9700 1 -- 172.21.15.181:0/125323857 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f250c031070 con 0x7f25181034f0 2024-05-03T20:31:26.594 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:26.594+0000 7f251db7a700 1 -- 172.21.15.181:0/125323857 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f25180638e0 con 0x7f25000b2880 2024-05-03T20:31:27.198 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:26 smithi181 ceph-mon[29801]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-03T20:31:27.199 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:26 smithi181 ceph-mon[29801]: osdmap e57: 9 total, 9 up, 9 in 2024-05-03T20:31:27.199 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:26 smithi181 ceph-mon[29801]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:31:27.199 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:26 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:31:27.199 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:26 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:31:27.199 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:26 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:27.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:26 smithi196 ceph-mon[32858]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-03T20:31:27.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:26 smithi196 ceph-mon[32858]: osdmap e57: 9 total, 9 up, 9 in 2024-05-03T20:31:27.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:26 smithi196 ceph-mon[32858]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:31:27.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:26 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:31:27.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:26 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:31:27.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:26 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:27.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:26 smithi204 ceph-mon[33595]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-03T20:31:27.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:26 smithi204 ceph-mon[33595]: osdmap e57: 9 total, 9 up, 9 in 2024-05-03T20:31:27.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:26 smithi204 ceph-mon[33595]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:31:27.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:26 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:31:27.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:26 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:31:27.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:26 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:28.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:27 smithi181 ceph-mon[29801]: osdmap e58: 9 total, 9 up, 9 in 2024-05-03T20:31:28.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:27 smithi196 ceph-mon[32858]: osdmap e58: 9 total, 9 up, 9 in 2024-05-03T20:31:28.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:27 smithi204 ceph-mon[33595]: osdmap e58: 9 total, 9 up, 9 in 2024-05-03T20:31:29.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:28 smithi181 ceph-mon[29801]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-03T20:31:29.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:28 smithi196 ceph-mon[32858]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-03T20:31:29.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:28 smithi204 ceph-mon[33595]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-03T20:31:29.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:29 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/1305088868' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96ce5aa6-e6a9-4d83-8562-589ac94c8cd1"}]: dispatch 2024-05-03T20:31:29.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:29 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/1305088868' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96ce5aa6-e6a9-4d83-8562-589ac94c8cd1"}]': finished 2024-05-03T20:31:29.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:29 smithi181 ceph-mon[29801]: osdmap e59: 10 total, 9 up, 10 in 2024-05-03T20:31:29.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:29 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T20:31:30.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:29 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/1305088868' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96ce5aa6-e6a9-4d83-8562-589ac94c8cd1"}]: dispatch 2024-05-03T20:31:30.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:29 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/1305088868' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96ce5aa6-e6a9-4d83-8562-589ac94c8cd1"}]': finished 2024-05-03T20:31:30.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:29 smithi196 ceph-mon[32858]: osdmap e59: 10 total, 9 up, 10 in 2024-05-03T20:31:30.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:29 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T20:31:30.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:29 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/1305088868' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96ce5aa6-e6a9-4d83-8562-589ac94c8cd1"}]: dispatch 2024-05-03T20:31:30.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:29 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/1305088868' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96ce5aa6-e6a9-4d83-8562-589ac94c8cd1"}]': finished 2024-05-03T20:31:30.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:29 smithi204 ceph-mon[33595]: osdmap e59: 10 total, 9 up, 10 in 2024-05-03T20:31:30.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:29 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T20:31:31.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:30 smithi181 ceph-mon[29801]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-03T20:31:31.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:30 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/4289753636' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:31:31.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:30 smithi196 ceph-mon[32858]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-03T20:31:31.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:30 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/4289753636' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:31:31.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:30 smithi204 ceph-mon[33595]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-03T20:31:31.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:30 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/4289753636' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:31:33.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:32 smithi181 ceph-mon[29801]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T20:31:33.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:32 smithi196 ceph-mon[32858]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T20:31:33.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:32 smithi204 ceph-mon[33595]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T20:31:35.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:34 smithi181 ceph-mon[29801]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-03T20:31:35.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:34 smithi196 ceph-mon[32858]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-03T20:31:35.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:34 smithi204 ceph-mon[33595]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-03T20:31:36.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:36 smithi181 ceph-mon[29801]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-03T20:31:37.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:36 smithi196 ceph-mon[32858]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-03T20:31:37.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:36 smithi204 ceph-mon[33595]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-03T20:31:38.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:37 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-03T20:31:38.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:37 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:38.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:37 smithi181 ceph-mon[29801]: Deploying daemon osd.9 on smithi181 2024-05-03T20:31:38.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:37 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-03T20:31:38.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:37 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:38.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:37 smithi196 ceph-mon[32858]: Deploying daemon osd.9 on smithi181 2024-05-03T20:31:38.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:37 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-03T20:31:38.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:37 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:38.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:37 smithi204 ceph-mon[33595]: Deploying daemon osd.9 on smithi181 2024-05-03T20:31:39.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:38 smithi181 ceph-mon[29801]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T20:31:39.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:38 smithi196 ceph-mon[32858]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T20:31:39.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:38 smithi204 ceph-mon[33595]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T20:31:41.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:40 smithi181 ceph-mon[29801]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-03T20:31:41.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:40 smithi196 ceph-mon[32858]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-03T20:31:41.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:40 smithi204 ceph-mon[33595]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-03T20:31:43.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:43 smithi196 ceph-mon[32858]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-03T20:31:43.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:43 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:43.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:43 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:43.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:43 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:31:43.870 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:43 smithi204 ceph-mon[33595]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-03T20:31:43.870 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:43 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:43.870 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:43 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:43.870 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:43 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:31:43.942 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:43 smithi181 ceph-mon[29801]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-03T20:31:43.942 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:43 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:43.942 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:43 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:43.942 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:43 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:31:45.802 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 9 on host 'smithi181' 2024-05-03T20:31:45.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:45.803+0000 7f2514ff9700 1 -- 172.21.15.181:0/125323857 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f25180638e0 con 0x7f25000b2880 2024-05-03T20:31:45.803 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:45 smithi181 ceph-mon[29801]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-03T20:31:45.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:45.806+0000 7f251db7a700 1 -- 172.21.15.181:0/125323857 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f25000b2880 msgr2=0x7f25000b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:31:45.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:45.806+0000 7f251db7a700 1 --2- 172.21.15.181:0/125323857 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f25000b2880 0x7f25000b4d40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2508005950 tx=0x7f250800b410 comp rx=0 tx=0).stop 2024-05-03T20:31:45.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:45.806+0000 7f251db7a700 1 -- 172.21.15.181:0/125323857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25181034f0 msgr2=0x7f2518199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:31:45.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:45.806+0000 7f251db7a700 1 --2- 172.21.15.181:0/125323857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25181034f0 0x7f2518199630 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f250c009f50 tx=0x7f250c0047b0 comp rx=0 tx=0).stop 2024-05-03T20:31:45.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:45.806+0000 7f251db7a700 1 -- 172.21.15.181:0/125323857 shutdown_connections 2024-05-03T20:31:45.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:45.806+0000 7f251db7a700 1 --2- 172.21.15.181:0/125323857 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f25000b2880 0x7f25000b4d40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:31:45.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:45.806+0000 7f251db7a700 1 --2- 172.21.15.181:0/125323857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2518105090 0x7f25181a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:31:45.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:45.806+0000 7f251db7a700 1 --2- 172.21.15.181:0/125323857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25181034f0 0x7f2518199630 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:31:45.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:45.806+0000 7f251db7a700 1 --2- 172.21.15.181:0/125323857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25181046f0 0x7f2518199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:31:45.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:45.806+0000 7f251db7a700 1 -- 172.21.15.181:0/125323857 >> 172.21.15.181:0/125323857 conn(0x7f25180fea90 msgr2=0x7f2518107a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:31:45.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:45.806+0000 7f251db7a700 1 -- 172.21.15.181:0/125323857 shutdown_connections 2024-05-03T20:31:45.806 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:31:45.806+0000 7f251db7a700 1 -- 172.21.15.181:0/125323857 wait complete. 2024-05-03T20:31:45.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:45 smithi196 ceph-mon[32858]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-03T20:31:45.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:45 smithi204 ceph-mon[33595]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-03T20:31:46.574 DEBUG:teuthology.orchestra.run.smithi181:osd.9> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.9.service 2024-05-03T20:31:46.577 INFO:tasks.cephadm:Deploying osd.10 on smithi196 with /dev/vg_nvme/lv_1... 2024-05-03T20:31:46.577 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-03T20:31:46.824 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T20:31:46.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:46 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:46.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:46 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:46.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:46 smithi196 ceph-mon[32858]: from='osd.9 [v2:172.21.15.181:6826/241693472,v1:172.21.15.181:6827/241693472]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-03T20:31:46.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:46 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:46.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:46 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:46.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:46 smithi181 ceph-mon[29801]: from='osd.9 [v2:172.21.15.181:6826/241693472,v1:172.21.15.181:6827/241693472]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-03T20:31:47.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:46 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:47.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:46 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:47.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:46 smithi204 ceph-mon[33595]: from='osd.9 [v2:172.21.15.181:6826/241693472,v1:172.21.15.181:6827/241693472]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-03T20:31:48.103 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:47 smithi181 ceph-mon[29801]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-03T20:31:48.103 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:47 smithi181 ceph-mon[29801]: from='osd.9 [v2:172.21.15.181:6826/241693472,v1:172.21.15.181:6827/241693472]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-03T20:31:48.103 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:47 smithi181 ceph-mon[29801]: osdmap e60: 10 total, 9 up, 10 in 2024-05-03T20:31:48.103 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:47 smithi181 ceph-mon[29801]: from='osd.9 [v2:172.21.15.181:6826/241693472,v1:172.21.15.181:6827/241693472]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-03T20:31:48.103 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:47 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T20:31:48.103 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:47 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:48.103 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:47 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:48.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:47 smithi196 ceph-mon[32858]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-03T20:31:48.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:47 smithi196 ceph-mon[32858]: from='osd.9 [v2:172.21.15.181:6826/241693472,v1:172.21.15.181:6827/241693472]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-03T20:31:48.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:47 smithi196 ceph-mon[32858]: osdmap e60: 10 total, 9 up, 10 in 2024-05-03T20:31:48.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:47 smithi196 ceph-mon[32858]: from='osd.9 [v2:172.21.15.181:6826/241693472,v1:172.21.15.181:6827/241693472]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-03T20:31:48.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:47 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T20:31:48.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:47 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:48.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:47 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:48.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:47 smithi204 ceph-mon[33595]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-03T20:31:48.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:47 smithi204 ceph-mon[33595]: from='osd.9 [v2:172.21.15.181:6826/241693472,v1:172.21.15.181:6827/241693472]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-03T20:31:48.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:47 smithi204 ceph-mon[33595]: osdmap e60: 10 total, 9 up, 10 in 2024-05-03T20:31:48.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:47 smithi204 ceph-mon[33595]: from='osd.9 [v2:172.21.15.181:6826/241693472,v1:172.21.15.181:6827/241693472]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-03T20:31:48.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:47 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T20:31:48.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:47 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:48.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:47 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:48.687 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-03T20:31:48.687 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-03T20:31:48.687 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-05-03T20:31:48.687 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-05-03T20:31:48.687 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012938 s, 810 MB/s 2024-05-03T20:31:48.687 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-05-03T20:31:49.005 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:48 smithi181 ceph-mon[29801]: from='osd.9 [v2:172.21.15.181:6826/241693472,v1:172.21.15.181:6827/241693472]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-03T20:31:49.006 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:48 smithi181 ceph-mon[29801]: osdmap e61: 10 total, 9 up, 10 in 2024-05-03T20:31:49.006 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T20:31:49.006 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:48 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T20:31:49.006 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:48 smithi181 ceph-mon[29801]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-03T20:31:49.006 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:48 smithi181 ceph-mon[29801]: from='osd.9 [v2:172.21.15.181:6826/241693472,v1:172.21.15.181:6827/241693472]' entity='osd.9' 2024-05-03T20:31:49.006 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:31:48 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:31:48.719+0000 7f9665be4700 -1 osd.9 0 waiting for initial osdmap 2024-05-03T20:31:49.006 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:31:48 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:31:48.886+0000 7f965d1ce700 -1 osd.9 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T20:31:49.062 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:48 smithi196 ceph-mon[32858]: from='osd.9 [v2:172.21.15.181:6826/241693472,v1:172.21.15.181:6827/241693472]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-03T20:31:49.062 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:48 smithi196 ceph-mon[32858]: osdmap e61: 10 total, 9 up, 10 in 2024-05-03T20:31:49.062 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T20:31:49.062 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:48 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T20:31:49.062 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:48 smithi196 ceph-mon[32858]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-03T20:31:49.062 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:48 smithi196 ceph-mon[32858]: from='osd.9 [v2:172.21.15.181:6826/241693472,v1:172.21.15.181:6827/241693472]' entity='osd.9' 2024-05-03T20:31:49.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:48 smithi204 ceph-mon[33595]: from='osd.9 [v2:172.21.15.181:6826/241693472,v1:172.21.15.181:6827/241693472]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-03T20:31:49.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:48 smithi204 ceph-mon[33595]: osdmap e61: 10 total, 9 up, 10 in 2024-05-03T20:31:49.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T20:31:49.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:48 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T20:31:49.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:48 smithi204 ceph-mon[33595]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-03T20:31:49.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:48 smithi204 ceph-mon[33595]: from='osd.9 [v2:172.21.15.181:6826/241693472,v1:172.21.15.181:6827/241693472]' entity='osd.9' 2024-05-03T20:31:49.376 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_1 2024-05-03T20:31:49.619 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T20:31:49.991 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:49 smithi196 ceph-mon[32858]: purged_snaps scrub starts 2024-05-03T20:31:49.992 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:49 smithi196 ceph-mon[32858]: purged_snaps scrub ok 2024-05-03T20:31:49.992 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:49 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T20:31:49.992 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:49 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T20:31:50.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:49 smithi204 ceph-mon[33595]: purged_snaps scrub starts 2024-05-03T20:31:50.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:49 smithi204 ceph-mon[33595]: purged_snaps scrub ok 2024-05-03T20:31:50.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:49 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T20:31:50.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:49 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T20:31:50.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:49 smithi181 ceph-mon[29801]: purged_snaps scrub starts 2024-05-03T20:31:50.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:49 smithi181 ceph-mon[29801]: purged_snaps scrub ok 2024-05-03T20:31:50.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:49 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T20:31:50.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:49 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T20:31:50.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:50 smithi181 ceph-mon[29801]: osd.9 [v2:172.21.15.181:6826/241693472,v1:172.21.15.181:6827/241693472] boot 2024-05-03T20:31:50.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:50 smithi181 ceph-mon[29801]: osdmap e62: 10 total, 10 up, 10 in 2024-05-03T20:31:50.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:50 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T20:31:50.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:50 smithi181 ceph-mon[29801]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-03T20:31:50.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:50 smithi181 ceph-mon[29801]: Detected new or changed devices on smithi181 2024-05-03T20:31:50.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:50 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:50.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:50 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:50.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:50 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:50.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:50 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:50.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:50 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:50.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:50 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:50.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:50 smithi181 ceph-mon[29801]: Adjusting osd_memory_target on smithi181 to 4272M 2024-05-03T20:31:50.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:50 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:50.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:50 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:50.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:50 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:31:50.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:50 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:51.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:50 smithi196 ceph-mon[32858]: osd.9 [v2:172.21.15.181:6826/241693472,v1:172.21.15.181:6827/241693472] boot 2024-05-03T20:31:51.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:50 smithi196 ceph-mon[32858]: osdmap e62: 10 total, 10 up, 10 in 2024-05-03T20:31:51.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:50 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T20:31:51.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:50 smithi196 ceph-mon[32858]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-03T20:31:51.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:50 smithi196 ceph-mon[32858]: Detected new or changed devices on smithi181 2024-05-03T20:31:51.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:50 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:51.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:50 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:51.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:50 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:51.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:50 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:51.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:50 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:51.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:50 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:51.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:50 smithi196 ceph-mon[32858]: Adjusting osd_memory_target on smithi181 to 4272M 2024-05-03T20:31:51.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:50 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:51.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:50 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:51.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:50 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:31:51.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:50 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:51.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:50 smithi204 ceph-mon[33595]: osd.9 [v2:172.21.15.181:6826/241693472,v1:172.21.15.181:6827/241693472] boot 2024-05-03T20:31:51.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:50 smithi204 ceph-mon[33595]: osdmap e62: 10 total, 10 up, 10 in 2024-05-03T20:31:51.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:50 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T20:31:51.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:50 smithi204 ceph-mon[33595]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-03T20:31:51.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:50 smithi204 ceph-mon[33595]: Detected new or changed devices on smithi181 2024-05-03T20:31:51.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:50 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:51.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:50 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:51.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:50 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:51.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:50 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:51.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:50 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:51.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:50 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:31:51.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:50 smithi204 ceph-mon[33595]: Adjusting osd_memory_target on smithi181 to 4272M 2024-05-03T20:31:51.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:50 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:51.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:50 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:51.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:50 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:31:51.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:50 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:51.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.129+0000 7f4d08581700 1 -- 172.21.15.196:0/636052508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d00103080 msgr2=0x7f4d0010a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:31:51.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.129+0000 7f4d08581700 1 --2- 172.21.15.196:0/636052508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d00103080 0x7f4d0010a940 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800b0a0 tx=0x7f4cf800b3b0 comp rx=0 tx=0).stop 2024-05-03T20:31:51.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.129+0000 7f4d08581700 1 -- 172.21.15.196:0/636052508 shutdown_connections 2024-05-03T20:31:51.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.129+0000 7f4d08581700 1 --2- 172.21.15.196:0/636052508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d00103080 0x7f4d0010a940 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:31:51.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.129+0000 7f4d08581700 1 --2- 172.21.15.196:0/636052508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d00103a30 0x7f4d00103e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:31:51.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.129+0000 7f4d08581700 1 --2- 172.21.15.196:0/636052508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d001026e0 0x7f4d00102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:31:51.128 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.129+0000 7f4d08581700 1 -- 172.21.15.196:0/636052508 >> 172.21.15.196:0/636052508 conn(0x7f4d000fd170 msgr2=0x7f4d000ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:31:51.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.129+0000 7f4d08581700 1 -- 172.21.15.196:0/636052508 shutdown_connections 2024-05-03T20:31:51.129 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.129+0000 7f4d08581700 1 -- 172.21.15.196:0/636052508 wait complete. 2024-05-03T20:31:51.131 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.130+0000 7f4d08581700 1 Processor -- start 2024-05-03T20:31:51.131 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.130+0000 7f4d08581700 1 -- start start 2024-05-03T20:31:51.131 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.130+0000 7f4d08581700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d001026e0 0x7f4d001995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:31:51.131 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.130+0000 7f4d08581700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d00103080 0x7f4d00199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:31:51.131 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.130+0000 7f4d08581700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d00103a30 0x7f4d001a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:31:51.131 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.130+0000 7f4d08581700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0010dee0 con 0x7f4d001026e0 2024-05-03T20:31:51.131 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.130+0000 7f4d08581700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0010dd60 con 0x7f4d00103a30 2024-05-03T20:31:51.131 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.130+0000 7f4d08581700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0010e060 con 0x7f4d00103080 2024-05-03T20:31:51.131 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.130+0000 7f4d06b1e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d00103a30 0x7f4d001a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:31:51.131 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.130+0000 7f4d06b1e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d00103a30 0x7f4d001a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:33308/0 (socket says 172.21.15.196:33308) 2024-05-03T20:31:51.131 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.130+0000 7f4d06b1e700 1 -- 172.21.15.196:0/3568740817 learned_addr learned my addr 172.21.15.196:0/3568740817 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-05-03T20:31:51.131 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.130+0000 7f4d0631d700 1 --2- 172.21.15.196:0/3568740817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d001026e0 0x7f4d001995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:31:51.131 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.130+0000 7f4d05b1c700 1 --2- 172.21.15.196:0/3568740817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d00103080 0x7f4d00199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:31:51.132 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.130+0000 7f4d0631d700 1 -- 172.21.15.196:0/3568740817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d00103080 msgr2=0x7f4d00199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:31:51.132 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.130+0000 7f4d0631d700 1 --2- 172.21.15.196:0/3568740817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d00103080 0x7f4d00199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:31:51.132 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.130+0000 7f4d0631d700 1 -- 172.21.15.196:0/3568740817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d00103a30 msgr2=0x7f4d001a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:31:51.132 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.130+0000 7f4d0631d700 1 --2- 172.21.15.196:0/3568740817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d00103a30 0x7f4d001a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:31:51.132 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.130+0000 7f4d0631d700 1 -- 172.21.15.196:0/3568740817 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d001a1240 con 0x7f4d001026e0 2024-05-03T20:31:51.132 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.130+0000 7f4d06b1e700 1 --2- 172.21.15.196:0/3568740817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d00103a30 0x7f4d001a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T20:31:51.132 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.130+0000 7f4d0631d700 1 --2- 172.21.15.196:0/3568740817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d001026e0 0x7f4d001995e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc00e9f0 tx=0x7f4cfc00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:31:51.132 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.131+0000 7f4cf77fe700 1 -- 172.21.15.196:0/3568740817 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc004220 con 0x7f4d001026e0 2024-05-03T20:31:51.132 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.131+0000 7f4d08581700 1 -- 172.21.15.196:0/3568740817 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d001a1530 con 0x7f4d001026e0 2024-05-03T20:31:51.132 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.131+0000 7f4cf77fe700 1 -- 172.21.15.196:0/3568740817 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cfc0043c0 con 0x7f4d001026e0 2024-05-03T20:31:51.132 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.131+0000 7f4cf77fe700 1 -- 172.21.15.196:0/3568740817 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc003be0 con 0x7f4d001026e0 2024-05-03T20:31:51.132 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.131+0000 7f4d08581700 1 -- 172.21.15.196:0/3568740817 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d001a1ac0 con 0x7f4d001026e0 2024-05-03T20:31:51.132 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.131+0000 7f4d08581700 1 -- 172.21.15.196:0/3568740817 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d0006a190 con 0x7f4d001026e0 2024-05-03T20:31:51.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.133+0000 7f4cf77fe700 1 -- 172.21.15.196:0/3568740817 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f4cfc00c7c0 con 0x7f4d001026e0 2024-05-03T20:31:51.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.133+0000 7f4cf77fe700 1 --2- 172.21.15.196:0/3568740817 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4cec0b2880 0x7f4cec0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:31:51.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.134+0000 7f4cf77fe700 1 -- 172.21.15.196:0/3568740817 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6261+0+0 (secure 0 0 0) 0x7f4cfc0e1d90 con 0x7f4d001026e0 2024-05-03T20:31:51.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.134+0000 7f4d05b1c700 1 --2- 172.21.15.196:0/3568740817 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4cec0b2880 0x7f4cec0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:31:51.133 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.134+0000 7f4d05b1c700 1 --2- 172.21.15.196:0/3568740817 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4cec0b2880 0x7f4cec0b4d40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0004630 tx=0x7f4cf000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:31:51.136 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.137+0000 7f4cf77fe700 1 -- 172.21.15.196:0/3568740817 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4cfc0aa2a0 con 0x7f4d001026e0 2024-05-03T20:31:51.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:31:51.345+0000 7f4d08581700 1 -- 172.21.15.196:0/3568740817 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4d000ff2f0 con 0x7f4cec0b2880 2024-05-03T20:31:52.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:51 smithi196 ceph-mon[32858]: osdmap e63: 10 total, 10 up, 10 in 2024-05-03T20:31:52.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:51 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:31:52.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:51 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:52.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:51 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:31:52.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:51 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:52.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:51 smithi196 ceph-mon[32858]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:31:52.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:51 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:31:52.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:51 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:31:52.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:51 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:52.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:51 smithi204 ceph-mon[33595]: osdmap e63: 10 total, 10 up, 10 in 2024-05-03T20:31:52.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:51 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:31:52.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:51 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:52.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:51 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:31:52.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:51 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:52.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:51 smithi204 ceph-mon[33595]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:31:52.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:51 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:31:52.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:51 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:31:52.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:51 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:52.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:51 smithi181 ceph-mon[29801]: osdmap e63: 10 total, 10 up, 10 in 2024-05-03T20:31:52.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:51 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:31:52.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:51 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:52.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:51 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:31:52.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:51 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:31:52.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:51 smithi181 ceph-mon[29801]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:31:52.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:51 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:31:52.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:51 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:31:52.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:51 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:31:52.985 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:52 smithi196 ceph-mon[32858]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:31:53.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:52 smithi204 ceph-mon[33595]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:31:53.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:52 smithi181 ceph-mon[29801]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:31:54.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:53 smithi181 ceph-mon[29801]: from='client.? 172.21.15.196:0/940558066' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6c9ffab-731c-46d5-88cd-5f21ba5acfa0"}]: dispatch 2024-05-03T20:31:54.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:53 smithi181 ceph-mon[29801]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6c9ffab-731c-46d5-88cd-5f21ba5acfa0"}]: dispatch 2024-05-03T20:31:54.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:53 smithi181 ceph-mon[29801]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6c9ffab-731c-46d5-88cd-5f21ba5acfa0"}]': finished 2024-05-03T20:31:54.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:53 smithi181 ceph-mon[29801]: osdmap e64: 11 total, 10 up, 11 in 2024-05-03T20:31:54.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:53 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T20:31:54.232 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:53 smithi196 ceph-mon[32858]: from='client.? 172.21.15.196:0/940558066' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6c9ffab-731c-46d5-88cd-5f21ba5acfa0"}]: dispatch 2024-05-03T20:31:54.232 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:53 smithi196 ceph-mon[32858]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6c9ffab-731c-46d5-88cd-5f21ba5acfa0"}]: dispatch 2024-05-03T20:31:54.232 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:53 smithi196 ceph-mon[32858]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6c9ffab-731c-46d5-88cd-5f21ba5acfa0"}]': finished 2024-05-03T20:31:54.232 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:53 smithi196 ceph-mon[32858]: osdmap e64: 11 total, 10 up, 11 in 2024-05-03T20:31:54.232 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:53 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T20:31:54.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:53 smithi204 ceph-mon[33595]: from='client.? 172.21.15.196:0/940558066' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6c9ffab-731c-46d5-88cd-5f21ba5acfa0"}]: dispatch 2024-05-03T20:31:54.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:53 smithi204 ceph-mon[33595]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6c9ffab-731c-46d5-88cd-5f21ba5acfa0"}]: dispatch 2024-05-03T20:31:54.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:53 smithi204 ceph-mon[33595]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6c9ffab-731c-46d5-88cd-5f21ba5acfa0"}]': finished 2024-05-03T20:31:54.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:53 smithi204 ceph-mon[33595]: osdmap e64: 11 total, 10 up, 11 in 2024-05-03T20:31:54.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:53 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T20:31:55.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:54 smithi181 ceph-mon[29801]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:31:55.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:54 smithi181 ceph-mon[29801]: from='client.? 172.21.15.196:0/2478118911' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:31:55.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:54 smithi196 ceph-mon[32858]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:31:55.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:54 smithi196 ceph-mon[32858]: from='client.? 172.21.15.196:0/2478118911' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:31:55.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:54 smithi204 ceph-mon[33595]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:31:55.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:54 smithi204 ceph-mon[33595]: from='client.? 172.21.15.196:0/2478118911' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:31:57.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:56 smithi181 ceph-mon[29801]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:31:57.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:56 smithi196 ceph-mon[32858]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:31:57.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:56 smithi204 ceph-mon[33595]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:31:59.208 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:31:58 smithi196 ceph-mon[32858]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:31:59.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:31:58 smithi181 ceph-mon[29801]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:31:59.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:31:58 smithi204 ceph-mon[33595]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:32:01.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:00 smithi196 ceph-mon[32858]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:32:01.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:00 smithi181 ceph-mon[29801]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:32:01.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:00 smithi204 ceph-mon[33595]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:32:02.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:01 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-03T20:32:02.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:01 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:02.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:01 smithi196 ceph-mon[32858]: Deploying daemon osd.10 on smithi196 2024-05-03T20:32:02.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:01 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-03T20:32:02.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:01 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:02.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:01 smithi181 ceph-mon[29801]: Deploying daemon osd.10 on smithi196 2024-05-03T20:32:02.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:01 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-03T20:32:02.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:01 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:02.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:01 smithi204 ceph-mon[33595]: Deploying daemon osd.10 on smithi196 2024-05-03T20:32:03.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:02 smithi181 ceph-mon[29801]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:32:03.230 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:02 smithi196 ceph-mon[32858]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:32:03.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:02 smithi204 ceph-mon[33595]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:32:05.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:04 smithi196 ceph-mon[32858]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:32:05.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:04 smithi181 ceph-mon[29801]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:32:05.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:04 smithi204 ceph-mon[33595]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:32:07.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:07 smithi181 ceph-mon[29801]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:32:07.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:07.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:07.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:32:07.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:07 smithi196 ceph-mon[32858]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:32:07.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:07.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:07.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:32:07.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:07 smithi204 ceph-mon[33595]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:32:07.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:07.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:07.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:32:09.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:09 smithi196 ceph-mon[32858]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:32:09.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:09 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:09.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:09 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:09.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:09 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:09.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:09 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:32:09.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:09 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:09.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:09 smithi181 ceph-mon[29801]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:32:09.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:09 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:09.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:09 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:09.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:09 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:09.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:09 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:32:09.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:09 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:09.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:09 smithi204 ceph-mon[33595]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:32:09.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:09 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:09.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:09 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:09.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:09 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:09.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:09 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:32:09.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:09 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:10.007 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:32:10.007+0000 7f4cf77fe700 1 -- 172.21.15.196:0/3568740817 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f4d000ff2f0 con 0x7f4cec0b2880 2024-05-03T20:32:10.007 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 10 on host 'smithi196' 2024-05-03T20:32:10.008 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:32:10.009+0000 7f4d08581700 1 -- 172.21.15.196:0/3568740817 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4cec0b2880 msgr2=0x7f4cec0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:10.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:32:10.009+0000 7f4d08581700 1 --2- 172.21.15.196:0/3568740817 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4cec0b2880 0x7f4cec0b4d40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0004630 tx=0x7f4cf000b3c0 comp rx=0 tx=0).stop 2024-05-03T20:32:10.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:32:10.009+0000 7f4d08581700 1 -- 172.21.15.196:0/3568740817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d001026e0 msgr2=0x7f4d001995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:10.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:32:10.009+0000 7f4d08581700 1 --2- 172.21.15.196:0/3568740817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d001026e0 0x7f4d001995e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc00e9f0 tx=0x7f4cfc00ed00 comp rx=0 tx=0).stop 2024-05-03T20:32:10.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:32:10.009+0000 7f4d08581700 1 -- 172.21.15.196:0/3568740817 shutdown_connections 2024-05-03T20:32:10.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:32:10.009+0000 7f4d08581700 1 --2- 172.21.15.196:0/3568740817 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4cec0b2880 0x7f4cec0b4d40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:10.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:32:10.009+0000 7f4d08581700 1 --2- 172.21.15.196:0/3568740817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d00103a30 0x7f4d001a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:10.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:32:10.009+0000 7f4d08581700 1 --2- 172.21.15.196:0/3568740817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d001026e0 0x7f4d001995e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:10.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:32:10.009+0000 7f4d08581700 1 --2- 172.21.15.196:0/3568740817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d00103080 0x7f4d00199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:10.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:32:10.009+0000 7f4d08581700 1 -- 172.21.15.196:0/3568740817 >> 172.21.15.196:0/3568740817 conn(0x7f4d000fd170 msgr2=0x7f4d000febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:32:10.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:32:10.009+0000 7f4d08581700 1 -- 172.21.15.196:0/3568740817 shutdown_connections 2024-05-03T20:32:10.009 INFO:teuthology.orchestra.run.smithi196.stderr:2024-05-03T20:32:10.009+0000 7f4d08581700 1 -- 172.21.15.196:0/3568740817 wait complete. 2024-05-03T20:32:10.619 DEBUG:teuthology.orchestra.run.smithi196:osd.10> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.10.service 2024-05-03T20:32:10.621 INFO:tasks.cephadm:Deploying osd.11 on smithi204 with /dev/vg_nvme/lv_1... 2024-05-03T20:32:10.621 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-03T20:32:10.869 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T20:32:11.356 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:10 smithi204 ceph-mon[33595]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:32:11.357 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:10 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:11.357 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:10 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:11.357 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:10 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:32:11.357 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:10 smithi204 ceph-mon[33595]: from='osd.10 [v2:172.21.15.196:6824/3980645995,v1:172.21.15.196:6825/3980645995]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T20:32:11.357 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:10 smithi204 ceph-mon[33595]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T20:32:11.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:10 smithi196 ceph-mon[32858]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:32:11.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:10 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:11.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:10 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:11.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:10 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:32:11.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:10 smithi196 ceph-mon[32858]: from='osd.10 [v2:172.21.15.196:6824/3980645995,v1:172.21.15.196:6825/3980645995]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T20:32:11.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:10 smithi196 ceph-mon[32858]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T20:32:11.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:10 smithi181 ceph-mon[29801]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:32:11.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:10 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:11.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:10 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:11.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:10 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:32:11.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:10 smithi181 ceph-mon[29801]: from='osd.10 [v2:172.21.15.196:6824/3980645995,v1:172.21.15.196:6825/3980645995]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T20:32:11.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:10 smithi181 ceph-mon[29801]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T20:32:12.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:12 smithi196 ceph-mon[32858]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-03T20:32:12.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:12 smithi196 ceph-mon[32858]: from='osd.10 [v2:172.21.15.196:6824/3980645995,v1:172.21.15.196:6825/3980645995]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-03T20:32:12.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:12 smithi196 ceph-mon[32858]: osdmap e65: 11 total, 10 up, 11 in 2024-05-03T20:32:12.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T20:32:12.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:12 smithi196 ceph-mon[32858]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-03T20:32:12.358 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:32:12 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:32:12.009+0000 7f48f400e700 -1 osd.10 0 waiting for initial osdmap 2024-05-03T20:32:12.358 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:32:12 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:32:12.022+0000 7f48efe01700 -1 osd.10 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T20:32:12.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:12 smithi204 ceph-mon[33595]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-03T20:32:12.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:12 smithi204 ceph-mon[33595]: from='osd.10 [v2:172.21.15.196:6824/3980645995,v1:172.21.15.196:6825/3980645995]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-03T20:32:12.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:12 smithi204 ceph-mon[33595]: osdmap e65: 11 total, 10 up, 11 in 2024-05-03T20:32:12.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T20:32:12.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:12 smithi204 ceph-mon[33595]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-03T20:32:12.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:12 smithi181 ceph-mon[29801]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-03T20:32:12.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:12 smithi181 ceph-mon[29801]: from='osd.10 [v2:172.21.15.196:6824/3980645995,v1:172.21.15.196:6825/3980645995]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-03T20:32:12.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:12 smithi181 ceph-mon[29801]: osdmap e65: 11 total, 10 up, 11 in 2024-05-03T20:32:12.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T20:32:12.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:12 smithi181 ceph-mon[29801]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-05-03T20:32:13.043 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-03T20:32:13.043 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-03T20:32:13.043 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-05-03T20:32:13.043 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-05-03T20:32:13.043 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011191 s, 937 MB/s 2024-05-03T20:32:13.043 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-05-03T20:32:13.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:13 smithi204 ceph-mon[33595]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:32:13.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:13 smithi204 ceph-mon[33595]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-03T20:32:13.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:13 smithi204 ceph-mon[33595]: osdmap e66: 11 total, 10 up, 11 in 2024-05-03T20:32:13.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:13 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T20:32:13.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:13 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T20:32:13.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:13 smithi196 ceph-mon[32858]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:32:13.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:13 smithi196 ceph-mon[32858]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-03T20:32:13.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:13 smithi196 ceph-mon[32858]: osdmap e66: 11 total, 10 up, 11 in 2024-05-03T20:32:13.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:13 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T20:32:13.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:13 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T20:32:13.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:13 smithi181 ceph-mon[29801]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T20:32:13.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:13 smithi181 ceph-mon[29801]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-05-03T20:32:13.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:13 smithi181 ceph-mon[29801]: osdmap e66: 11 total, 10 up, 11 in 2024-05-03T20:32:13.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:13 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T20:32:13.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:13 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T20:32:13.738 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_1 2024-05-03T20:32:13.988 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T20:32:14.307 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:14 smithi204 ceph-mon[33595]: purged_snaps scrub starts 2024-05-03T20:32:14.307 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:14 smithi204 ceph-mon[33595]: purged_snaps scrub ok 2024-05-03T20:32:14.307 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:14 smithi204 ceph-mon[33595]: osd.10 [v2:172.21.15.196:6824/3980645995,v1:172.21.15.196:6825/3980645995] boot 2024-05-03T20:32:14.307 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:14 smithi204 ceph-mon[33595]: osdmap e67: 11 total, 11 up, 11 in 2024-05-03T20:32:14.307 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:14 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T20:32:14.307 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:14 smithi204 ceph-mon[33595]: Detected new or changed devices on smithi196 2024-05-03T20:32:14.307 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:14 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:14.307 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:14 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:14.308 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:14 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:32:14.308 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:14 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:32:14.308 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:14 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:32:14.308 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:14 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:32:14.308 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:14 smithi204 ceph-mon[33595]: Adjusting osd_memory_target on smithi196 to 4272M 2024-05-03T20:32:14.308 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:14 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:14.308 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:14 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:14.308 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:14 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:32:14.308 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:14 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:14.308 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:14 smithi204 ceph-mon[33595]: osdmap e68: 11 total, 11 up, 11 in 2024-05-03T20:32:14.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:14 smithi196 ceph-mon[32858]: purged_snaps scrub starts 2024-05-03T20:32:14.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:14 smithi196 ceph-mon[32858]: purged_snaps scrub ok 2024-05-03T20:32:14.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:14 smithi196 ceph-mon[32858]: osd.10 [v2:172.21.15.196:6824/3980645995,v1:172.21.15.196:6825/3980645995] boot 2024-05-03T20:32:14.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:14 smithi196 ceph-mon[32858]: osdmap e67: 11 total, 11 up, 11 in 2024-05-03T20:32:14.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:14 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T20:32:14.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:14 smithi196 ceph-mon[32858]: Detected new or changed devices on smithi196 2024-05-03T20:32:14.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:14 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:14.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:14 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:14.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:14 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:32:14.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:14 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:32:14.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:14 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:32:14.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:14 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:32:14.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:14 smithi196 ceph-mon[32858]: Adjusting osd_memory_target on smithi196 to 4272M 2024-05-03T20:32:14.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:14 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:14.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:14 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:14.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:14 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:32:14.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:14 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:14.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:14 smithi196 ceph-mon[32858]: osdmap e68: 11 total, 11 up, 11 in 2024-05-03T20:32:14.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:14 smithi181 ceph-mon[29801]: purged_snaps scrub starts 2024-05-03T20:32:14.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:14 smithi181 ceph-mon[29801]: purged_snaps scrub ok 2024-05-03T20:32:14.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:14 smithi181 ceph-mon[29801]: osd.10 [v2:172.21.15.196:6824/3980645995,v1:172.21.15.196:6825/3980645995] boot 2024-05-03T20:32:14.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:14 smithi181 ceph-mon[29801]: osdmap e67: 11 total, 11 up, 11 in 2024-05-03T20:32:14.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:14 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T20:32:14.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:14 smithi181 ceph-mon[29801]: Detected new or changed devices on smithi196 2024-05-03T20:32:14.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:14 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:14.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:14 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:14.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:14 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:32:14.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:14 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:32:14.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:14 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:32:14.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:14 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:32:14.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:14 smithi181 ceph-mon[29801]: Adjusting osd_memory_target on smithi196 to 4272M 2024-05-03T20:32:14.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:14 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:14.464 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:14 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:14.465 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:14 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:32:14.465 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:14 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:14.465 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:14 smithi181 ceph-mon[29801]: osdmap e68: 11 total, 11 up, 11 in 2024-05-03T20:32:15.718 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.717+0000 7fc96016b700 1 -- 172.21.15.204:0/3830869706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc958102dd0 msgr2=0x7fc958103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:15.718 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.717+0000 7fc96016b700 1 --2- 172.21.15.204:0/3830869706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc958102dd0 0x7fc958103230 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc948009990 tx=0x7fc948009ca0 comp rx=0 tx=0).stop 2024-05-03T20:32:15.718 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.717+0000 7fc96016b700 1 -- 172.21.15.204:0/3830869706 shutdown_connections 2024-05-03T20:32:15.718 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.717+0000 7fc96016b700 1 --2- 172.21.15.204:0/3830869706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc958103770 0x7fc95810a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:15.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.717+0000 7fc96016b700 1 --2- 172.21.15.204:0/3830869706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc958101bd0 0x7fc958101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:15.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.717+0000 7fc96016b700 1 --2- 172.21.15.204:0/3830869706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc958102dd0 0x7fc958103230 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:15.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.717+0000 7fc96016b700 1 -- 172.21.15.204:0/3830869706 >> 172.21.15.204:0/3830869706 conn(0x7fc9580fd170 msgr2=0x7fc9580ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:32:15.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.717+0000 7fc96016b700 1 -- 172.21.15.204:0/3830869706 shutdown_connections 2024-05-03T20:32:15.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.717+0000 7fc96016b700 1 -- 172.21.15.204:0/3830869706 wait complete. 2024-05-03T20:32:15.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.717+0000 7fc96016b700 1 Processor -- start 2024-05-03T20:32:15.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.718+0000 7fc96016b700 1 -- start start 2024-05-03T20:32:15.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.718+0000 7fc96016b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc958101bd0 0x7fc95810b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:15.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.718+0000 7fc96016b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc958102dd0 0x7fc95819e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:15.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.718+0000 7fc96016b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc958103770 0x7fc9581a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:15.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.718+0000 7fc96016b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc95810df40 con 0x7fc958102dd0 2024-05-03T20:32:15.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.718+0000 7fc96016b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc95810ddc0 con 0x7fc958103770 2024-05-03T20:32:15.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.718+0000 7fc96016b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc95810e0c0 con 0x7fc958101bd0 2024-05-03T20:32:15.719 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.718+0000 7fc95df07700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc958101bd0 0x7fc95810b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:15.720 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.718+0000 7fc95df07700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc958101bd0 0x7fc95810b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:35668/0 (socket says 172.21.15.204:35668) 2024-05-03T20:32:15.720 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.718+0000 7fc95df07700 1 -- 172.21.15.204:0/3150145751 learned_addr learned my addr 172.21.15.204:0/3150145751 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-03T20:32:15.720 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.718+0000 7fc95e708700 1 --2- 172.21.15.204:0/3150145751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc958103770 0x7fc9581a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:15.720 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.718+0000 7fc95d706700 1 --2- 172.21.15.204:0/3150145751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc958102dd0 0x7fc95819e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:15.720 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.718+0000 7fc95df07700 1 -- 172.21.15.204:0/3150145751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc958103770 msgr2=0x7fc9581a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:15.720 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.718+0000 7fc95df07700 1 --2- 172.21.15.204:0/3150145751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc958103770 0x7fc9581a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:15.720 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.718+0000 7fc95df07700 1 -- 172.21.15.204:0/3150145751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc958102dd0 msgr2=0x7fc95819e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:15.720 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.718+0000 7fc95df07700 1 --2- 172.21.15.204:0/3150145751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc958102dd0 0x7fc95819e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:15.720 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.718+0000 7fc95df07700 1 -- 172.21.15.204:0/3150145751 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9581a12d0 con 0x7fc958101bd0 2024-05-03T20:32:15.720 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.718+0000 7fc95d706700 1 --2- 172.21.15.204:0/3150145751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc958102dd0 0x7fc95819e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:32:15.720 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.718+0000 7fc95df07700 1 --2- 172.21.15.204:0/3150145751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc958101bd0 0x7fc95810b590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc95400d980 tx=0x7fc95400dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:32:15.720 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.718+0000 7fc94effd700 1 -- 172.21.15.204:0/3150145751 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954017030 con 0x7fc958101bd0 2024-05-03T20:32:15.720 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.718+0000 7fc94effd700 1 -- 172.21.15.204:0/3150145751 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9540044d0 con 0x7fc958101bd0 2024-05-03T20:32:15.720 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.718+0000 7fc96016b700 1 -- 172.21.15.204:0/3150145751 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9581a15c0 con 0x7fc958101bd0 2024-05-03T20:32:15.720 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.718+0000 7fc94effd700 1 -- 172.21.15.204:0/3150145751 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954003aa0 con 0x7fc958101bd0 2024-05-03T20:32:15.721 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.718+0000 7fc96016b700 1 -- 172.21.15.204:0/3150145751 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9581a1b00 con 0x7fc958101bd0 2024-05-03T20:32:15.721 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.719+0000 7fc96016b700 1 -- 172.21.15.204:0/3150145751 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc95806a190 con 0x7fc958101bd0 2024-05-03T20:32:15.721 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.721+0000 7fc94effd700 1 -- 172.21.15.204:0/3150145751 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fc95401a070 con 0x7fc958101bd0 2024-05-03T20:32:15.721 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.721+0000 7fc94effd700 1 --2- 172.21.15.204:0/3150145751 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fc9440b2880 0x7fc9440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:15.722 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.721+0000 7fc94effd700 1 -- 172.21.15.204:0/3150145751 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6694+0+0 (secure 0 0 0) 0x7fc9540e19a0 con 0x7fc958101bd0 2024-05-03T20:32:15.722 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.722+0000 7fc95d706700 1 --2- 172.21.15.204:0/3150145751 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fc9440b2880 0x7fc9440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:15.722 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.722+0000 7fc95d706700 1 --2- 172.21.15.204:0/3150145751 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fc9440b2880 0x7fc9440b4d40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc948006010 tx=0x7fc948009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:32:15.724 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.725+0000 7fc94effd700 1 -- 172.21.15.204:0/3150145751 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc9540a9d10 con 0x7fc958101bd0 2024-05-03T20:32:15.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:15 smithi196 ceph-mon[32858]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-03T20:32:15.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:15 smithi196 ceph-mon[32858]: osdmap e69: 11 total, 11 up, 11 in 2024-05-03T20:32:15.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:15 smithi204 ceph-mon[33595]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-03T20:32:15.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:15 smithi204 ceph-mon[33595]: osdmap e69: 11 total, 11 up, 11 in 2024-05-03T20:32:15.934 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:15.935+0000 7fc96016b700 1 -- 172.21.15.204:0/3150145751 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fc9580638e0 con 0x7fc9440b2880 2024-05-03T20:32:15.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:15 smithi181 ceph-mon[29801]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-03T20:32:15.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:15 smithi181 ceph-mon[29801]: osdmap e69: 11 total, 11 up, 11 in 2024-05-03T20:32:16.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:16 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:32:16.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:16 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:32:16.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:16 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:16.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:16 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:32:16.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:16 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:32:16.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:16 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:16.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:16 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T20:32:16.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:16 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T20:32:16.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:16 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:17.803 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:17 smithi204 ceph-mon[33595]: pgmap v230: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T20:32:17.803 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:17 smithi204 ceph-mon[33595]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:32:17.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:17 smithi196 ceph-mon[32858]: pgmap v230: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T20:32:17.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:17 smithi196 ceph-mon[32858]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:32:17.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:17 smithi181 ceph-mon[29801]: pgmap v230: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T20:32:17.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:17 smithi181 ceph-mon[29801]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:32:19.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:19 smithi196 ceph-mon[32858]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T20:32:19.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:19 smithi196 ceph-mon[32858]: from='client.? 172.21.15.204:0/2699781774' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f58a2d6-bbbb-4e4d-bf83-2c886dd9484b"}]: dispatch 2024-05-03T20:32:19.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:19 smithi196 ceph-mon[32858]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f58a2d6-bbbb-4e4d-bf83-2c886dd9484b"}]: dispatch 2024-05-03T20:32:19.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:19 smithi196 ceph-mon[32858]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f58a2d6-bbbb-4e4d-bf83-2c886dd9484b"}]': finished 2024-05-03T20:32:19.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:19 smithi196 ceph-mon[32858]: osdmap e70: 12 total, 11 up, 12 in 2024-05-03T20:32:19.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:19 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T20:32:19.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:19 smithi196 ceph-mon[32858]: from='client.? 172.21.15.204:0/1059452484' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:32:19.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:19 smithi204 ceph-mon[33595]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T20:32:19.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:19 smithi204 ceph-mon[33595]: from='client.? 172.21.15.204:0/2699781774' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f58a2d6-bbbb-4e4d-bf83-2c886dd9484b"}]: dispatch 2024-05-03T20:32:19.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:19 smithi204 ceph-mon[33595]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f58a2d6-bbbb-4e4d-bf83-2c886dd9484b"}]: dispatch 2024-05-03T20:32:19.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:19 smithi204 ceph-mon[33595]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f58a2d6-bbbb-4e4d-bf83-2c886dd9484b"}]': finished 2024-05-03T20:32:19.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:19 smithi204 ceph-mon[33595]: osdmap e70: 12 total, 11 up, 12 in 2024-05-03T20:32:19.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T20:32:19.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:19 smithi204 ceph-mon[33595]: from='client.? 172.21.15.204:0/1059452484' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:32:19.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:19 smithi181 ceph-mon[29801]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T20:32:19.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:19 smithi181 ceph-mon[29801]: from='client.? 172.21.15.204:0/2699781774' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f58a2d6-bbbb-4e4d-bf83-2c886dd9484b"}]: dispatch 2024-05-03T20:32:19.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:19 smithi181 ceph-mon[29801]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f58a2d6-bbbb-4e4d-bf83-2c886dd9484b"}]: dispatch 2024-05-03T20:32:19.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:19 smithi181 ceph-mon[29801]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f58a2d6-bbbb-4e4d-bf83-2c886dd9484b"}]': finished 2024-05-03T20:32:19.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:19 smithi181 ceph-mon[29801]: osdmap e70: 12 total, 11 up, 12 in 2024-05-03T20:32:19.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:19 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T20:32:19.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:19 smithi181 ceph-mon[29801]: from='client.? 172.21.15.204:0/1059452484' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T20:32:21.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:21 smithi204 ceph-mon[33595]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T20:32:21.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:21 smithi181 ceph-mon[29801]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T20:32:22.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:21 smithi196 ceph-mon[32858]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T20:32:23.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:23 smithi181 ceph-mon[29801]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-03T20:32:23.984 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:23 smithi204 ceph-mon[33595]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-03T20:32:24.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:23 smithi196 ceph-mon[32858]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-03T20:32:25.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:25 smithi181 ceph-mon[29801]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-03T20:32:25.990 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:25 smithi204 ceph-mon[33595]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-03T20:32:26.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:25 smithi196 ceph-mon[32858]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-03T20:32:26.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:26 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-03T20:32:26.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:26 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:26.961 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:26 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-03T20:32:26.961 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:26 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:26.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:26 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-03T20:32:26.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:26 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:27.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:27 smithi181 ceph-mon[29801]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T20:32:27.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:27 smithi181 ceph-mon[29801]: Deploying daemon osd.11 on smithi204 2024-05-03T20:32:27.991 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:27 smithi204 ceph-mon[33595]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T20:32:27.991 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:27 smithi204 ceph-mon[33595]: Deploying daemon osd.11 on smithi204 2024-05-03T20:32:28.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:27 smithi196 ceph-mon[32858]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T20:32:28.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:27 smithi196 ceph-mon[32858]: Deploying daemon osd.11 on smithi204 2024-05-03T20:32:29.802 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:29 smithi204 ceph-mon[33595]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T20:32:29.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:29 smithi181 ceph-mon[29801]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T20:32:30.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:29 smithi196 ceph-mon[32858]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T20:32:31.928 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:31 smithi204 ceph-mon[33595]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-03T20:32:31.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:31 smithi181 ceph-mon[29801]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-03T20:32:32.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:31 smithi196 ceph-mon[32858]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-03T20:32:33.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:32 smithi196 ceph-mon[32858]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-03T20:32:33.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:32 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:33.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:32 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:33.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:32 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:32:33.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:32 smithi204 ceph-mon[33595]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-03T20:32:33.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:32 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:33.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:32 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:33.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:32 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:32:33.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:32 smithi181 ceph-mon[29801]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-03T20:32:33.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:32 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:33.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:32 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:33.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:32 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:32:35.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:34 smithi196 ceph-mon[32858]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T20:32:35.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:34 smithi204 ceph-mon[33595]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T20:32:35.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:34 smithi181 ceph-mon[29801]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T20:32:35.901 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 11 on host 'smithi204' 2024-05-03T20:32:35.902 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:35.902+0000 7fc94effd700 1 -- 172.21.15.204:0/3150145751 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fc9580638e0 con 0x7fc9440b2880 2024-05-03T20:32:35.904 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:35.905+0000 7fc96016b700 1 -- 172.21.15.204:0/3150145751 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fc9440b2880 msgr2=0x7fc9440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:35.904 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:35.905+0000 7fc96016b700 1 --2- 172.21.15.204:0/3150145751 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fc9440b2880 0x7fc9440b4d40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc948006010 tx=0x7fc948009510 comp rx=0 tx=0).stop 2024-05-03T20:32:35.904 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:35.905+0000 7fc96016b700 1 -- 172.21.15.204:0/3150145751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc958101bd0 msgr2=0x7fc95810b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:35.904 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:35.905+0000 7fc96016b700 1 --2- 172.21.15.204:0/3150145751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc958101bd0 0x7fc95810b590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc95400d980 tx=0x7fc95400dd40 comp rx=0 tx=0).stop 2024-05-03T20:32:35.904 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:35.905+0000 7fc96016b700 1 -- 172.21.15.204:0/3150145751 shutdown_connections 2024-05-03T20:32:35.904 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:35.905+0000 7fc96016b700 1 --2- 172.21.15.204:0/3150145751 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fc9440b2880 0x7fc9440b4d40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:35.904 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:35.905+0000 7fc96016b700 1 --2- 172.21.15.204:0/3150145751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc958103770 0x7fc9581a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:35.904 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:35.905+0000 7fc96016b700 1 --2- 172.21.15.204:0/3150145751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc958101bd0 0x7fc95810b590 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:35.905 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:35.905+0000 7fc96016b700 1 --2- 172.21.15.204:0/3150145751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc958102dd0 0x7fc95819e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:35.905 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:35.905+0000 7fc96016b700 1 -- 172.21.15.204:0/3150145751 >> 172.21.15.204:0/3150145751 conn(0x7fc9580fd170 msgr2=0x7fc9581060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:32:35.905 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:35.905+0000 7fc96016b700 1 -- 172.21.15.204:0/3150145751 shutdown_connections 2024-05-03T20:32:35.905 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:35.905+0000 7fc96016b700 1 -- 172.21.15.204:0/3150145751 wait complete. 2024-05-03T20:32:36.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:36.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:36.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:36.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:32:36.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:36.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:36.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:36.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:36 smithi204 ceph-mon[33595]: from='osd.11 [v2:172.21.15.204:6824/488710574,v1:172.21.15.204:6825/488710574]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T20:32:36.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:36 smithi204 ceph-mon[33595]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T20:32:36.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:32:36.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:36.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:36.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:36.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:32:36.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:36.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:36.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:36.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:36 smithi196 ceph-mon[32858]: from='osd.11 [v2:172.21.15.204:6824/488710574,v1:172.21.15.204:6825/488710574]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T20:32:36.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:36 smithi196 ceph-mon[32858]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T20:32:36.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:32:36.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:36.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:36.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:36.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:32:36.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:36.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:36.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:36.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:36 smithi181 ceph-mon[29801]: from='osd.11 [v2:172.21.15.204:6824/488710574,v1:172.21.15.204:6825/488710574]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T20:32:36.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:36 smithi181 ceph-mon[29801]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T20:32:36.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:32:36.586 DEBUG:teuthology.orchestra.run.smithi204:osd.11> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.11.service 2024-05-03T20:32:36.588 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-03T20:32:36.588 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-03T20:32:36.837 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:32:37.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:37 smithi181 ceph-mon[29801]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T20:32:37.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:37 smithi181 ceph-mon[29801]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-03T20:32:37.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:37 smithi181 ceph-mon[29801]: from='osd.11 [v2:172.21.15.204:6824/488710574,v1:172.21.15.204:6825/488710574]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-03T20:32:37.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:37 smithi181 ceph-mon[29801]: osdmap e71: 12 total, 11 up, 12 in 2024-05-03T20:32:37.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:37 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T20:32:37.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:37 smithi181 ceph-mon[29801]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-03T20:32:37.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:37 smithi196 ceph-mon[32858]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T20:32:37.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:37 smithi196 ceph-mon[32858]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-03T20:32:37.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:37 smithi196 ceph-mon[32858]: from='osd.11 [v2:172.21.15.204:6824/488710574,v1:172.21.15.204:6825/488710574]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-03T20:32:37.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:37 smithi196 ceph-mon[32858]: osdmap e71: 12 total, 11 up, 12 in 2024-05-03T20:32:37.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:37 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T20:32:37.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:37 smithi196 ceph-mon[32858]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-03T20:32:37.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:37 smithi204 ceph-mon[33595]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T20:32:37.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:37 smithi204 ceph-mon[33595]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-03T20:32:37.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:37 smithi204 ceph-mon[33595]: from='osd.11 [v2:172.21.15.204:6824/488710574,v1:172.21.15.204:6825/488710574]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-03T20:32:37.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:37 smithi204 ceph-mon[33595]: osdmap e71: 12 total, 11 up, 12 in 2024-05-03T20:32:37.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:37 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T20:32:37.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:37 smithi204 ceph-mon[33595]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-05-03T20:32:37.994 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:32:37 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:32:37.860+0000 7f0c1565d700 -1 osd.11 0 waiting for initial osdmap 2024-05-03T20:32:37.994 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:32:37 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:32:37.874+0000 7f0c11450700 -1 osd.11 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T20:32:38.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.468+0000 7f7f1aae1700 1 -- 172.21.15.181:0/2009318235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f14103fb0 msgr2=0x7f7f1410a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:38.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.468+0000 7f7f1aae1700 1 --2- 172.21.15.181:0/2009318235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f14103fb0 0x7f7f1410a860 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00b0d0 tx=0x7f7f0c00b3e0 comp rx=0 tx=0).stop 2024-05-03T20:32:38.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.468+0000 7f7f1aae1700 1 -- 172.21.15.181:0/2009318235 shutdown_connections 2024-05-03T20:32:38.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.468+0000 7f7f1aae1700 1 --2- 172.21.15.181:0/2009318235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f14103fb0 0x7f7f1410a860 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:38.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.468+0000 7f7f1aae1700 1 --2- 172.21.15.181:0/2009318235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f14102410 0x7f7f14102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:38.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.468+0000 7f7f1aae1700 1 --2- 172.21.15.181:0/2009318235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f14103610 0x7f7f14103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:38.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.468+0000 7f7f1aae1700 1 -- 172.21.15.181:0/2009318235 >> 172.21.15.181:0/2009318235 conn(0x7f7f140fd9d0 msgr2=0x7f7f140ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:32:38.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.468+0000 7f7f1aae1700 1 -- 172.21.15.181:0/2009318235 shutdown_connections 2024-05-03T20:32:38.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.468+0000 7f7f1aae1700 1 -- 172.21.15.181:0/2009318235 wait complete. 2024-05-03T20:32:38.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.469+0000 7f7f1aae1700 1 Processor -- start 2024-05-03T20:32:38.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.469+0000 7f7f1aae1700 1 -- start start 2024-05-03T20:32:38.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.469+0000 7f7f1aae1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f14102410 0x7f7f14199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:38.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.469+0000 7f7f1aae1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f14103610 0x7f7f14199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:38.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.469+0000 7f7f1aae1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f14103fb0 0x7f7f141a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:38.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.469+0000 7f7f1aae1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1410dea0 con 0x7f7f14103fb0 2024-05-03T20:32:38.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.469+0000 7f7f1aae1700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1410dd20 con 0x7f7f14102410 2024-05-03T20:32:38.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.469+0000 7f7f1aae1700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1410e020 con 0x7f7f14103610 2024-05-03T20:32:38.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.469+0000 7f7f13fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f14103610 0x7f7f14199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:38.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.469+0000 7f7f1907e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f14103fb0 0x7f7f141a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:38.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.469+0000 7f7f13fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f14103610 0x7f7f14199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.181:47712/0 (socket says 172.21.15.181:47712) 2024-05-03T20:32:38.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.469+0000 7f7f13fff700 1 -- 172.21.15.181:0/3851463106 learned_addr learned my addr 172.21.15.181:0/3851463106 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:32:38.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.469+0000 7f7f1887d700 1 --2- 172.21.15.181:0/3851463106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f14102410 0x7f7f14199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:38.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.470+0000 7f7f13fff700 1 -- 172.21.15.181:0/3851463106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f14102410 msgr2=0x7f7f14199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:38.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.470+0000 7f7f13fff700 1 --2- 172.21.15.181:0/3851463106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f14102410 0x7f7f14199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:38.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.470+0000 7f7f13fff700 1 -- 172.21.15.181:0/3851463106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f14103fb0 msgr2=0x7f7f141a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:38.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.470+0000 7f7f13fff700 1 --2- 172.21.15.181:0/3851463106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f14103fb0 0x7f7f141a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:38.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.470+0000 7f7f13fff700 1 -- 172.21.15.181:0/3851463106 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f141a1250 con 0x7f7f14103610 2024-05-03T20:32:38.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.470+0000 7f7f1907e700 1 --2- 172.21.15.181:0/3851463106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f14103fb0 0x7f7f141a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T20:32:38.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.470+0000 7f7f13fff700 1 --2- 172.21.15.181:0/3851463106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f14103610 0x7f7f14199b80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000c9e0 tx=0x7f7f0000ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:32:38.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.470+0000 7f7f11ffb700 1 -- 172.21.15.181:0/3851463106 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f00007ac0 con 0x7f7f14103610 2024-05-03T20:32:38.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.470+0000 7f7f1aae1700 1 -- 172.21.15.181:0/3851463106 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f141a1540 con 0x7f7f14103610 2024-05-03T20:32:38.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.470+0000 7f7f11ffb700 1 -- 172.21.15.181:0/3851463106 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f00007c60 con 0x7f7f14103610 2024-05-03T20:32:38.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.470+0000 7f7f11ffb700 1 -- 172.21.15.181:0/3851463106 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f00018660 con 0x7f7f14103610 2024-05-03T20:32:38.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.470+0000 7f7f1aae1700 1 -- 172.21.15.181:0/3851463106 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f141a1ad0 con 0x7f7f14103610 2024-05-03T20:32:38.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.471+0000 7f7f1aae1700 1 -- 172.21.15.181:0/3851463106 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f1406a190 con 0x7f7f14103610 2024-05-03T20:32:38.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.472+0000 7f7f11ffb700 1 -- 172.21.15.181:0/3851463106 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f7f0000f450 con 0x7f7f14103610 2024-05-03T20:32:38.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.473+0000 7f7f11ffb700 1 --2- 172.21.15.181:0/3851463106 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f7f040b2880 0x7f7f040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:38.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.473+0000 7f7f1887d700 1 --2- 172.21.15.181:0/3851463106 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f7f040b2880 0x7f7f040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:38.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.473+0000 7f7f11ffb700 1 -- 172.21.15.181:0/3851463106 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6847+0+0 (secure 0 0 0) 0x7f7f000e1d60 con 0x7f7f14103610 2024-05-03T20:32:38.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.473+0000 7f7f1887d700 1 --2- 172.21.15.181:0/3851463106 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f7f040b2880 0x7f7f040b4d40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7f14103470 tx=0x7f7f08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:32:38.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.476+0000 7f7f11ffb700 1 -- 172.21.15.181:0/3851463106 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7f00014030 con 0x7f7f14103610 2024-05-03T20:32:38.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.677+0000 7f7f1aae1700 1 -- 172.21.15.181:0/3851463106 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f7f141a1f00 con 0x7f7f14103610 2024-05-03T20:32:38.678 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.678+0000 7f7f11ffb700 1 -- 172.21.15.181:0/3851463106 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7f7f000a9fa0 con 0x7f7f14103610 2024-05-03T20:32:38.678 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:32:38.680 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.680+0000 7f7f1aae1700 1 -- 172.21.15.181:0/3851463106 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f7f040b2880 msgr2=0x7f7f040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:38.680 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.680+0000 7f7f1aae1700 1 --2- 172.21.15.181:0/3851463106 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f7f040b2880 0x7f7f040b4d40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7f14103470 tx=0x7f7f08009260 comp rx=0 tx=0).stop 2024-05-03T20:32:38.680 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.680+0000 7f7f1aae1700 1 -- 172.21.15.181:0/3851463106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f14103610 msgr2=0x7f7f14199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:38.680 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.680+0000 7f7f1aae1700 1 --2- 172.21.15.181:0/3851463106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f14103610 0x7f7f14199b80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000c9e0 tx=0x7f7f0000ccf0 comp rx=0 tx=0).stop 2024-05-03T20:32:38.680 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.680+0000 7f7f1aae1700 1 -- 172.21.15.181:0/3851463106 shutdown_connections 2024-05-03T20:32:38.680 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.680+0000 7f7f1aae1700 1 --2- 172.21.15.181:0/3851463106 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f7f040b2880 0x7f7f040b4d40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:38.680 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.680+0000 7f7f1aae1700 1 --2- 172.21.15.181:0/3851463106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f14103fb0 0x7f7f141a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:38.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.680+0000 7f7f1aae1700 1 --2- 172.21.15.181:0/3851463106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f14102410 0x7f7f14199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:38.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.680+0000 7f7f1aae1700 1 --2- 172.21.15.181:0/3851463106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f14103610 0x7f7f14199b80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:38.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.680+0000 7f7f1aae1700 1 -- 172.21.15.181:0/3851463106 >> 172.21.15.181:0/3851463106 conn(0x7f7f140fd9d0 msgr2=0x7f7f1406bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:32:38.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.681+0000 7f7f1aae1700 1 -- 172.21.15.181:0/3851463106 shutdown_connections 2024-05-03T20:32:38.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:38.681+0000 7f7f1aae1700 1 -- 172.21.15.181:0/3851463106 wait complete. 2024-05-03T20:32:38.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:38 smithi181 ceph-mon[29801]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T20:32:38.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:38 smithi181 ceph-mon[29801]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-03T20:32:38.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:38 smithi181 ceph-mon[29801]: osdmap e72: 12 total, 11 up, 12 in 2024-05-03T20:32:38.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:38 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T20:32:38.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:38 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T20:32:38.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:38 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3851463106' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-03T20:32:39.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:38 smithi204 ceph-mon[33595]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T20:32:39.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:38 smithi204 ceph-mon[33595]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-03T20:32:39.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:38 smithi204 ceph-mon[33595]: osdmap e72: 12 total, 11 up, 12 in 2024-05-03T20:32:39.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:38 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T20:32:39.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:38 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T20:32:39.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:38 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3851463106' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-03T20:32:39.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:38 smithi196 ceph-mon[32858]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T20:32:39.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:38 smithi196 ceph-mon[32858]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-05-03T20:32:39.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:38 smithi196 ceph-mon[32858]: osdmap e72: 12 total, 11 up, 12 in 2024-05-03T20:32:39.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:38 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T20:32:39.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:38 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T20:32:39.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:38 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3851463106' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-03T20:32:39.375 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1714768333,"num_in_osds":12,"osd_in_since":1714768338,"num_remapped_pgs":0} 2024-05-03T20:32:39.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:39 smithi181 ceph-mon[29801]: purged_snaps scrub starts 2024-05-03T20:32:39.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:39 smithi181 ceph-mon[29801]: purged_snaps scrub ok 2024-05-03T20:32:39.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:39 smithi181 ceph-mon[29801]: osd.11 [v2:172.21.15.204:6824/488710574,v1:172.21.15.204:6825/488710574] boot 2024-05-03T20:32:39.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:39 smithi181 ceph-mon[29801]: osdmap e73: 12 total, 12 up, 12 in 2024-05-03T20:32:39.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:39 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T20:32:39.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:39 smithi181 ceph-mon[29801]: Detected new or changed devices on smithi204 2024-05-03T20:32:39.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:39 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:39.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:39 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:39.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:39 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:32:39.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:39 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:32:39.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:39 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:32:39.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:39 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:32:39.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:39 smithi181 ceph-mon[29801]: Adjusting osd_memory_target on smithi204 to 4272M 2024-05-03T20:32:39.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:39 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:39.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:39 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:39.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:39 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:32:39.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:39 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:40.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:39 smithi196 ceph-mon[32858]: purged_snaps scrub starts 2024-05-03T20:32:40.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:39 smithi196 ceph-mon[32858]: purged_snaps scrub ok 2024-05-03T20:32:40.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:39 smithi196 ceph-mon[32858]: osd.11 [v2:172.21.15.204:6824/488710574,v1:172.21.15.204:6825/488710574] boot 2024-05-03T20:32:40.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:39 smithi196 ceph-mon[32858]: osdmap e73: 12 total, 12 up, 12 in 2024-05-03T20:32:40.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:39 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T20:32:40.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:39 smithi196 ceph-mon[32858]: Detected new or changed devices on smithi204 2024-05-03T20:32:40.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:39 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:40.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:39 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:40.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:39 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:32:40.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:39 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:32:40.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:39 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:32:40.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:39 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:32:40.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:39 smithi196 ceph-mon[32858]: Adjusting osd_memory_target on smithi204 to 4272M 2024-05-03T20:32:40.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:39 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:40.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:39 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:40.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:39 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:32:40.358 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:39 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:40.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:39 smithi204 ceph-mon[33595]: purged_snaps scrub starts 2024-05-03T20:32:40.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:39 smithi204 ceph-mon[33595]: purged_snaps scrub ok 2024-05-03T20:32:40.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:39 smithi204 ceph-mon[33595]: osd.11 [v2:172.21.15.204:6824/488710574,v1:172.21.15.204:6825/488710574] boot 2024-05-03T20:32:40.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:39 smithi204 ceph-mon[33595]: osdmap e73: 12 total, 12 up, 12 in 2024-05-03T20:32:40.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:39 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T20:32:40.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:39 smithi204 ceph-mon[33595]: Detected new or changed devices on smithi204 2024-05-03T20:32:40.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:39 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:40.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:39 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:40.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:39 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:32:40.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:39 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:32:40.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:39 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:32:40.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:39 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:32:40.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:39 smithi204 ceph-mon[33595]: Adjusting osd_memory_target on smithi204 to 4272M 2024-05-03T20:32:40.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:39 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:40.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:39 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:40.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:39 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:32:40.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:39 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:40.376 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-03T20:32:40.616 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:32:41.041 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:40 smithi181 ceph-mon[29801]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:41.041 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:40 smithi181 ceph-mon[29801]: osdmap e74: 12 total, 12 up, 12 in 2024-05-03T20:32:41.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:40 smithi196 ceph-mon[32858]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:41.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:40 smithi196 ceph-mon[32858]: osdmap e74: 12 total, 12 up, 12 in 2024-05-03T20:32:41.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:40 smithi204 ceph-mon[33595]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:41.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:40 smithi204 ceph-mon[33595]: osdmap e74: 12 total, 12 up, 12 in 2024-05-03T20:32:42.218 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.217+0000 7f51338ff700 1 -- 172.21.15.181:0/3411919474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f512c102410 msgr2=0x7f512c102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:42.218 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.217+0000 7f51338ff700 1 --2- 172.21.15.181:0/3411919474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f512c102410 0x7f512c102830 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f51240099c0 tx=0x7f5124009cd0 comp rx=0 tx=0).stop 2024-05-03T20:32:42.219 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.217+0000 7f51338ff700 1 -- 172.21.15.181:0/3411919474 shutdown_connections 2024-05-03T20:32:42.219 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.217+0000 7f51338ff700 1 --2- 172.21.15.181:0/3411919474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f512c103fb0 0x7f512c10a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:42.219 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.217+0000 7f51338ff700 1 --2- 172.21.15.181:0/3411919474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f512c102410 0x7f512c102830 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:42.219 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.217+0000 7f51338ff700 1 --2- 172.21.15.181:0/3411919474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f512c103610 0x7f512c103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:42.219 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.217+0000 7f51338ff700 1 -- 172.21.15.181:0/3411919474 >> 172.21.15.181:0/3411919474 conn(0x7f512c0fd9d0 msgr2=0x7f512c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:32:42.219 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.217+0000 7f51338ff700 1 -- 172.21.15.181:0/3411919474 shutdown_connections 2024-05-03T20:32:42.219 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.217+0000 7f51338ff700 1 -- 172.21.15.181:0/3411919474 wait complete. 2024-05-03T20:32:42.219 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.217+0000 7f51338ff700 1 Processor -- start 2024-05-03T20:32:42.219 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.217+0000 7f51338ff700 1 -- start start 2024-05-03T20:32:42.219 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.218+0000 7f51338ff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f512c102410 0x7f512c1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:42.219 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.218+0000 7f51338ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f512c103610 0x7f512c199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:42.219 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.218+0000 7f51338ff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f512c103fb0 0x7f512c1a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:42.220 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.218+0000 7f51338ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f512c10df80 con 0x7f512c103610 2024-05-03T20:32:42.220 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.218+0000 7f51338ff700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f512c10de00 con 0x7f512c102410 2024-05-03T20:32:42.220 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.218+0000 7f51338ff700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f512c10e100 con 0x7f512c103fb0 2024-05-03T20:32:42.220 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.218+0000 7f5130e9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f512c103610 0x7f512c199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:42.220 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.218+0000 7f5130e9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f512c103610 0x7f512c199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41890/0 (socket says 172.21.15.181:41890) 2024-05-03T20:32:42.220 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.218+0000 7f5130e9a700 1 -- 172.21.15.181:0/308421073 learned_addr learned my addr 172.21.15.181:0/308421073 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:32:42.220 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.218+0000 7f513169b700 1 --2- 172.21.15.181:0/308421073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f512c102410 0x7f512c1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:42.220 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.218+0000 7f5130e9a700 1 -- 172.21.15.181:0/308421073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f512c103fb0 msgr2=0x7f512c1a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:42.220 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.218+0000 7f5131e9c700 1 --2- 172.21.15.181:0/308421073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f512c103fb0 0x7f512c1a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:42.220 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.218+0000 7f5130e9a700 1 --2- 172.21.15.181:0/308421073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f512c103fb0 0x7f512c1a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:42.220 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.218+0000 7f5130e9a700 1 -- 172.21.15.181:0/308421073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f512c102410 msgr2=0x7f512c1997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:42.220 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.218+0000 7f5130e9a700 1 --2- 172.21.15.181:0/308421073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f512c102410 0x7f512c1997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:42.220 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.218+0000 7f5130e9a700 1 -- 172.21.15.181:0/308421073 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512c1a1420 con 0x7f512c103610 2024-05-03T20:32:42.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.218+0000 7f513169b700 1 --2- 172.21.15.181:0/308421073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f512c102410 0x7f512c1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:32:42.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.218+0000 7f5131e9c700 1 --2- 172.21.15.181:0/308421073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f512c103fb0 0x7f512c1a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:32:42.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.218+0000 7f5130e9a700 1 --2- 172.21.15.181:0/308421073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f512c103610 0x7f512c199d00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f511c00b560 tx=0x7f511c00b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:32:42.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.218+0000 7f51227fc700 1 -- 172.21.15.181:0/308421073 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c015020 con 0x7f512c103610 2024-05-03T20:32:42.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.218+0000 7f51338ff700 1 -- 172.21.15.181:0/308421073 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f512c1a1710 con 0x7f512c103610 2024-05-03T20:32:42.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.218+0000 7f51227fc700 1 -- 172.21.15.181:0/308421073 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f511c011ce0 con 0x7f512c103610 2024-05-03T20:32:42.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.219+0000 7f51227fc700 1 -- 172.21.15.181:0/308421073 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c010da0 con 0x7f512c103610 2024-05-03T20:32:42.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.219+0000 7f51338ff700 1 -- 172.21.15.181:0/308421073 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f512c1a1cd0 con 0x7f512c103610 2024-05-03T20:32:42.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.220+0000 7f51338ff700 1 -- 172.21.15.181:0/308421073 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f512c06a190 con 0x7f512c103610 2024-05-03T20:32:42.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.221+0000 7f51227fc700 1 -- 172.21.15.181:0/308421073 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f511c018020 con 0x7f512c103610 2024-05-03T20:32:42.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.221+0000 7f51227fc700 1 --2- 172.21.15.181:0/308421073 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f51180b2880 0x7f51180b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:42.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.222+0000 7f513169b700 1 --2- 172.21.15.181:0/308421073 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f51180b2880 0x7f51180b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:42.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.222+0000 7f51227fc700 1 -- 172.21.15.181:0/308421073 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f511c0e1b80 con 0x7f512c103610 2024-05-03T20:32:42.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.222+0000 7f513169b700 1 --2- 172.21.15.181:0/308421073 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f51180b2880 0x7f51180b4d40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f51240038c0 tx=0x7f5124009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:32:42.224 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.225+0000 7f51227fc700 1 -- 172.21.15.181:0/308421073 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f511c01f020 con 0x7f512c103610 2024-05-03T20:32:42.432 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.432+0000 7f51338ff700 1 -- 172.21.15.181:0/308421073 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f512c1a2150 con 0x7f512c103610 2024-05-03T20:32:42.432 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.432+0000 7f51227fc700 1 -- 172.21.15.181:0/308421073 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7f511c0a9d40 con 0x7f512c103610 2024-05-03T20:32:42.432 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:32:42.435 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.435+0000 7f51338ff700 1 -- 172.21.15.181:0/308421073 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f51180b2880 msgr2=0x7f51180b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:42.435 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.435+0000 7f51338ff700 1 --2- 172.21.15.181:0/308421073 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f51180b2880 0x7f51180b4d40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f51240038c0 tx=0x7f5124009f10 comp rx=0 tx=0).stop 2024-05-03T20:32:42.435 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.436+0000 7f51338ff700 1 -- 172.21.15.181:0/308421073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f512c103610 msgr2=0x7f512c199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:42.435 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.436+0000 7f51338ff700 1 --2- 172.21.15.181:0/308421073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f512c103610 0x7f512c199d00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f511c00b560 tx=0x7f511c00b920 comp rx=0 tx=0).stop 2024-05-03T20:32:42.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.436+0000 7f51338ff700 1 -- 172.21.15.181:0/308421073 shutdown_connections 2024-05-03T20:32:42.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.436+0000 7f51338ff700 1 --2- 172.21.15.181:0/308421073 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f51180b2880 0x7f51180b4d40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:42.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.436+0000 7f51338ff700 1 --2- 172.21.15.181:0/308421073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f512c103fb0 0x7f512c1a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:42.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.436+0000 7f51338ff700 1 --2- 172.21.15.181:0/308421073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f512c102410 0x7f512c1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:42.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.436+0000 7f51338ff700 1 --2- 172.21.15.181:0/308421073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f512c103610 0x7f512c199d00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:42.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.436+0000 7f51338ff700 1 -- 172.21.15.181:0/308421073 >> 172.21.15.181:0/308421073 conn(0x7f512c0fd9d0 msgr2=0x7f512c06bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:32:42.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.436+0000 7f51338ff700 1 -- 172.21.15.181:0/308421073 shutdown_connections 2024-05-03T20:32:42.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:42.436+0000 7f51338ff700 1 -- 172.21.15.181:0/308421073 wait complete. 2024-05-03T20:32:43.064 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:42 smithi181 ceph-mon[29801]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:43.064 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:42 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/308421073' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-03T20:32:43.091 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":74,"num_osds":12,"num_up_osds":12,"osd_up_since":1714768358,"num_in_osds":12,"osd_in_since":1714768338,"num_remapped_pgs":0} 2024-05-03T20:32:43.092 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T20:32:43.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:42 smithi204 ceph-mon[33595]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:43.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:42 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/308421073' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-03T20:32:43.326 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:32:43.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:42 smithi196 ceph-mon[32858]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:43.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:42 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/308421073' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-03T20:32:44.970 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.956+0000 7f8bd8082700 1 -- 172.21.15.181:0/3519632029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bd0104100 msgr2=0x7f8bd0108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:44.970 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.956+0000 7f8bd8082700 1 --2- 172.21.15.181:0/3519632029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bd0104100 0x7f8bd0108120 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8bc00099c0 tx=0x7f8bc0009cd0 comp rx=0 tx=0).stop 2024-05-03T20:32:44.970 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.956+0000 7f8bd8082700 1 -- 172.21.15.181:0/3519632029 shutdown_connections 2024-05-03T20:32:44.971 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.956+0000 7f8bd8082700 1 --2- 172.21.15.181:0/3519632029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bd0108850 0x7f8bd01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:44.971 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.956+0000 7f8bd8082700 1 --2- 172.21.15.181:0/3519632029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bd0103750 0x7f8bd0103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:44.971 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.956+0000 7f8bd8082700 1 --2- 172.21.15.181:0/3519632029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bd0104100 0x7f8bd0108120 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:44.971 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.956+0000 7f8bd8082700 1 -- 172.21.15.181:0/3519632029 >> 172.21.15.181:0/3519632029 conn(0x7f8bd007a600 msgr2=0x7f8bd007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:32:44.971 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.957+0000 7f8bd8082700 1 -- 172.21.15.181:0/3519632029 shutdown_connections 2024-05-03T20:32:44.971 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.957+0000 7f8bd8082700 1 -- 172.21.15.181:0/3519632029 wait complete. 2024-05-03T20:32:44.971 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.957+0000 7f8bd8082700 1 Processor -- start 2024-05-03T20:32:44.971 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.957+0000 7f8bd8082700 1 -- start start 2024-05-03T20:32:44.971 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.958+0000 7f8bd8082700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bd0103750 0x7f8bd019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:44.971 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.958+0000 7f8bd8082700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bd0104100 0x7f8bd019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:44.971 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.958+0000 7f8bd8082700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bd0108850 0x7f8bd01998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:44.971 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.958+0000 7f8bd8082700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd0113630 con 0x7f8bd0108850 2024-05-03T20:32:44.971 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.958+0000 7f8bd8082700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd01134b0 con 0x7f8bd0103750 2024-05-03T20:32:44.972 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.958+0000 7f8bd8082700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd01137b0 con 0x7f8bd0104100 2024-05-03T20:32:44.972 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.958+0000 7f8bd661f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bd0108850 0x7f8bd01998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:44.972 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.958+0000 7f8bd661f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bd0108850 0x7f8bd01998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41912/0 (socket says 172.21.15.181:41912) 2024-05-03T20:32:44.972 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.958+0000 7f8bd661f700 1 -- 172.21.15.181:0/29393834 learned_addr learned my addr 172.21.15.181:0/29393834 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:32:44.972 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.958+0000 7f8bd561d700 1 --2- 172.21.15.181:0/29393834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bd0104100 0x7f8bd019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:44.972 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.958+0000 7f8bd5e1e700 1 --2- 172.21.15.181:0/29393834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bd0103750 0x7f8bd019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:44.972 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.958+0000 7f8bd661f700 1 -- 172.21.15.181:0/29393834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bd0104100 msgr2=0x7f8bd019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:44.972 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.958+0000 7f8bd661f700 1 --2- 172.21.15.181:0/29393834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bd0104100 0x7f8bd019fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:44.972 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.958+0000 7f8bd661f700 1 -- 172.21.15.181:0/29393834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bd0103750 msgr2=0x7f8bd019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:44.972 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.958+0000 7f8bd661f700 1 --2- 172.21.15.181:0/29393834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bd0103750 0x7f8bd019f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:44.972 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.958+0000 7f8bd661f700 1 -- 172.21.15.181:0/29393834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd019a010 con 0x7f8bd0108850 2024-05-03T20:32:44.972 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.958+0000 7f8bd661f700 1 --2- 172.21.15.181:0/29393834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bd0108850 0x7f8bd01998b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800ef30 tx=0x7f8bc800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:32:44.972 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.958+0000 7f8bc6ffd700 1 -- 172.21.15.181:0/29393834 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc80092e0 con 0x7f8bd0108850 2024-05-03T20:32:44.973 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.958+0000 7f8bd8082700 1 -- 172.21.15.181:0/29393834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd019a2a0 con 0x7f8bd0108850 2024-05-03T20:32:44.973 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.958+0000 7f8bc6ffd700 1 -- 172.21.15.181:0/29393834 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bc8004bb0 con 0x7f8bd0108850 2024-05-03T20:32:44.973 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.958+0000 7f8bc6ffd700 1 -- 172.21.15.181:0/29393834 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8004280 con 0x7f8bd0108850 2024-05-03T20:32:44.973 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.958+0000 7f8bd8082700 1 -- 172.21.15.181:0/29393834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd01a65c0 con 0x7f8bd0108850 2024-05-03T20:32:44.973 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.959+0000 7f8bd8082700 1 -- 172.21.15.181:0/29393834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd006a190 con 0x7f8bd0108850 2024-05-03T20:32:44.973 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.960+0000 7f8bc6ffd700 1 -- 172.21.15.181:0/29393834 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f8bc801b030 con 0x7f8bd0108850 2024-05-03T20:32:44.973 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.961+0000 7f8bc6ffd700 1 --2- 172.21.15.181:0/29393834 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8bbc0b2880 0x7f8bbc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:44.973 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.961+0000 7f8bd5e1e700 1 --2- 172.21.15.181:0/29393834 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8bbc0b2880 0x7f8bbc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:44.973 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.961+0000 7f8bc6ffd700 1 -- 172.21.15.181:0/29393834 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f8bc80e1860 con 0x7f8bd0108850 2024-05-03T20:32:44.973 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.961+0000 7f8bd5e1e700 1 --2- 172.21.15.181:0/29393834 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8bbc0b2880 0x7f8bbc0b4d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8bd019a9f0 tx=0x7f8bcc005d70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:32:44.973 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:44.966+0000 7f8bc6ffd700 1 -- 172.21.15.181:0/29393834 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8bc8022020 con 0x7f8bd0108850 2024-05-03T20:32:45.158 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:44 smithi196 ceph-mon[32858]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:45.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:45.176+0000 7f8bd8082700 1 -- 172.21.15.181:0/29393834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8bd002e510 con 0x7f8bd0108850 2024-05-03T20:32:45.175 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:44 smithi181 ceph-mon[29801]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:45.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:45.177+0000 7f8bc6ffd700 1 -- 172.21.15.181:0/29393834 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+18477 (secure 0 0 0) 0x7f8bc8068e20 con 0x7f8bd0108850 2024-05-03T20:32:45.177 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:32:45.177 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":74,"fsid":"1cd058ee-098b-11ef-bc97-c7b262605968","created":"2024-05-03T20:25:39.427815+0000","modified":"2024-05-03T20:32:39.862172+0000","last_up_change":"2024-05-03T20:32:38.854730+0000","last_in_change":"2024-05-03T20:32:18.787388+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T20:29:01.855187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5f594d97-10a4-4f77-aebc-c00dbe92a4af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6803","nonce":1683973278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6805","nonce":1683973278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6809","nonce":1683973278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6807","nonce":1683973278}]},"public_addr":"172.21.15.181:6803/1683973278","cluster_addr":"172.21.15.181:6805/1683973278","heartbeat_back_addr":"172.21.15.181:6809/1683973278","heartbeat_front_addr":"172.21.15.181:6807/1683973278","state":["exists","up"]},{"osd":1,"uuid":"9726d641-8930-4b45-9192-6ee2b6bfcdcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6801","nonce":2136873375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6803","nonce":2136873375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6807","nonce":2136873375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6805","nonce":2136873375}]},"public_addr":"172.21.15.196:6801/2136873375","cluster_addr":"172.21.15.196:6803/2136873375","heartbeat_back_addr":"172.21.15.196:6807/2136873375","heartbeat_front_addr":"172.21.15.196:6805/2136873375","state":["exists","up"]},{"osd":2,"uuid":"3d5573cc-37c1-4c3b-a61e-2363ef3ff964","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6801","nonce":3291673137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6803","nonce":3291673137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6807","nonce":3291673137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6805","nonce":3291673137}]},"public_addr":"172.21.15.204:6801/3291673137","cluster_addr":"172.21.15.204:6803/3291673137","heartbeat_back_addr":"172.21.15.204:6807/3291673137","heartbeat_front_addr":"172.21.15.204:6805/3291673137","state":["exists","up"]},{"osd":3,"uuid":"b9b205e3-a335-4771-9a66-55cfe6a8c31a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6811","nonce":29064971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6813","nonce":29064971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6817","nonce":29064971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6815","nonce":29064971}]},"public_addr":"172.21.15.181:6811/29064971","cluster_addr":"172.21.15.181:6813/29064971","heartbeat_back_addr":"172.21.15.181:6817/29064971","heartbeat_front_addr":"172.21.15.181:6815/29064971","state":["exists","up"]},{"osd":4,"uuid":"550f01ef-1947-4f9f-b759-34e3ea831839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6809","nonce":1145408995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6811","nonce":1145408995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6815","nonce":1145408995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6813","nonce":1145408995}]},"public_addr":"172.21.15.196:6809/1145408995","cluster_addr":"172.21.15.196:6811/1145408995","heartbeat_back_addr":"172.21.15.196:6815/1145408995","heartbeat_front_addr":"172.21.15.196:6813/1145408995","state":["exists","up"]},{"osd":5,"uuid":"6b74cb3a-f39a-4131-8793-1d5286856aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6809","nonce":855680559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6811","nonce":855680559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6815","nonce":855680559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6813","nonce":855680559}]},"public_addr":"172.21.15.204:6809/855680559","cluster_addr":"172.21.15.204:6811/855680559","heartbeat_back_addr":"172.21.15.204:6815/855680559","heartbeat_front_addr":"172.21.15.204:6813/855680559","state":["exists","up"]},{"osd":6,"uuid":"dbe19bdb-08eb-40aa-842b-0710963d4497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6819","nonce":1561969338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6821","nonce":1561969338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6825","nonce":1561969338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6823","nonce":1561969338}]},"public_addr":"172.21.15.181:6819/1561969338","cluster_addr":"172.21.15.181:6821/1561969338","heartbeat_back_addr":"172.21.15.181:6825/1561969338","heartbeat_front_addr":"172.21.15.181:6823/1561969338","state":["exists","up"]},{"osd":7,"uuid":"016c145c-ad67-42ac-9775-ca2840790abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6817","nonce":824890163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6819","nonce":824890163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6823","nonce":824890163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6821","nonce":824890163}]},"public_addr":"172.21.15.196:6817/824890163","cluster_addr":"172.21.15.196:6819/824890163","heartbeat_back_addr":"172.21.15.196:6823/824890163","heartbeat_front_addr":"172.21.15.196:6821/824890163","state":["exists","up"]},{"osd":8,"uuid":"10235638-c1f1-41d3-8e24-f8b2344bfeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6817","nonce":3566909552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6819","nonce":3566909552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6823","nonce":3566909552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6821","nonce":3566909552}]},"public_addr":"172.21.15.204:6817/3566909552","cluster_addr":"172.21.15.204:6819/3566909552","heartbeat_back_addr":"172.21.15.204:6823/3566909552","heartbeat_front_addr":"172.21.15.204:6821/3566909552","state":["exists","up"]},{"osd":9,"uuid":"96ce5aa6-e6a9-4d83-8562-589ac94c8cd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6827","nonce":241693472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6829","nonce":241693472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6833","nonce":241693472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6831","nonce":241693472}]},"public_addr":"172.21.15.181:6827/241693472","cluster_addr":"172.21.15.181:6829/241693472","heartbeat_back_addr":"172.21.15.181:6833/241693472","heartbeat_front_addr":"172.21.15.181:6831/241693472","state":["exists","up"]},{"osd":10,"uuid":"d6c9ffab-731c-46d5-88cd-5f21ba5acfa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6825","nonce":3980645995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6827","nonce":3980645995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6831","nonce":3980645995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6829","nonce":3980645995}]},"public_addr":"172.21.15.196:6825/3980645995","cluster_addr":"172.21.15.196:6827/3980645995","heartbeat_back_addr":"172.21.15.196:6831/3980645995","heartbeat_front_addr":"172.21.15.196:6829/3980645995","state":["exists","up"]},{"osd":11,"uuid":"3f58a2d6-bbbb-4e4d-bf83-2c886dd9484b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6825","nonce":488710574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6827","nonce":488710574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6831","nonce":488710574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6829","nonce":488710574}]},"public_addr":"172.21.15.204:6825/488710574","cluster_addr":"172.21.15.204:6827/488710574","heartbeat_back_addr":"172.21.15.204:6831/488710574","heartbeat_front_addr":"172.21.15.204:6829/488710574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:28:09.700091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:28:33.838568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:28:57.265536+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:29:21.720066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:29:45.480447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:30:09.558384+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:30:34.253331+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:30:57.986111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:31:22.470746+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:31:47.548762+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:32:11.285657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:32:36.888004+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6800/2389231557":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/3398276284":"2024-05-04T20:26:06.561713+0000","172.21.15.181:6801/2389231557":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/2327733609":"2024-05-04T20:26:06.561713+0000","172.21.15.181:0/628540759":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/3387163329":"2024-05-04T20:26:06.561713+0000","172.21.15.181:6800/1802984663":"2024-05-04T20:26:06.561713+0000","172.21.15.181:0/67924109":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/3693389657":"2024-05-04T20:26:33.766772+0000","172.21.15.181:6801/1802984663":"2024-05-04T20:26:06.561713+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T20:32:45.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:45.180+0000 7f8bd8082700 1 -- 172.21.15.181:0/29393834 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8bbc0b2880 msgr2=0x7f8bbc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:45.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:45.180+0000 7f8bd8082700 1 --2- 172.21.15.181:0/29393834 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8bbc0b2880 0x7f8bbc0b4d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8bd019a9f0 tx=0x7f8bcc005d70 comp rx=0 tx=0).stop 2024-05-03T20:32:45.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:45.180+0000 7f8bd8082700 1 -- 172.21.15.181:0/29393834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bd0108850 msgr2=0x7f8bd01998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:45.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:45.180+0000 7f8bd8082700 1 --2- 172.21.15.181:0/29393834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bd0108850 0x7f8bd01998b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800ef30 tx=0x7f8bc800c3a0 comp rx=0 tx=0).stop 2024-05-03T20:32:45.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:45.180+0000 7f8bd8082700 1 -- 172.21.15.181:0/29393834 shutdown_connections 2024-05-03T20:32:45.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:45.180+0000 7f8bd8082700 1 --2- 172.21.15.181:0/29393834 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8bbc0b2880 0x7f8bbc0b4d40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:45.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:45.180+0000 7f8bd8082700 1 --2- 172.21.15.181:0/29393834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bd0108850 0x7f8bd01998b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:45.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:45.180+0000 7f8bd8082700 1 --2- 172.21.15.181:0/29393834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bd0103750 0x7f8bd019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:45.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:45.180+0000 7f8bd8082700 1 --2- 172.21.15.181:0/29393834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bd0104100 0x7f8bd019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:45.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:45.180+0000 7f8bd8082700 1 -- 172.21.15.181:0/29393834 >> 172.21.15.181:0/29393834 conn(0x7f8bd007a600 msgr2=0x7f8bd0101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:32:45.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:45.180+0000 7f8bd8082700 1 -- 172.21.15.181:0/29393834 shutdown_connections 2024-05-03T20:32:45.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:45.180+0000 7f8bd8082700 1 -- 172.21.15.181:0/29393834 wait complete. 2024-05-03T20:32:45.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:44 smithi204 ceph-mon[33595]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:45.878 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-03T20:29:01.855187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-03T20:32:45.879 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-03T20:32:46.114 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:32:46.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:45 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/29393834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T20:32:46.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:45 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/29393834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T20:32:46.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:45 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/29393834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T20:32:47.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:46 smithi181 ceph-mon[29801]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:47.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:46 smithi196 ceph-mon[32858]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:47.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:46 smithi204 ceph-mon[33595]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:47.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.579+0000 7f3a618cf700 1 -- 172.21.15.181:0/317988617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a5c104100 msgr2=0x7f3a5c108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:47.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.579+0000 7f3a618cf700 1 --2- 172.21.15.181:0/317988617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a5c104100 0x7f3a5c108120 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3a500099c0 tx=0x7f3a50009cd0 comp rx=0 tx=0).stop 2024-05-03T20:32:47.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.579+0000 7f3a618cf700 1 -- 172.21.15.181:0/317988617 shutdown_connections 2024-05-03T20:32:47.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.579+0000 7f3a618cf700 1 --2- 172.21.15.181:0/317988617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a5c108850 0x7f3a5c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:47.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.579+0000 7f3a618cf700 1 --2- 172.21.15.181:0/317988617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a5c103750 0x7f3a5c103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:47.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.579+0000 7f3a618cf700 1 --2- 172.21.15.181:0/317988617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a5c104100 0x7f3a5c108120 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:47.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.579+0000 7f3a618cf700 1 -- 172.21.15.181:0/317988617 >> 172.21.15.181:0/317988617 conn(0x7f3a5c07a600 msgr2=0x7f3a5c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:32:47.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.579+0000 7f3a618cf700 1 -- 172.21.15.181:0/317988617 shutdown_connections 2024-05-03T20:32:47.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.579+0000 7f3a618cf700 1 -- 172.21.15.181:0/317988617 wait complete. 2024-05-03T20:32:47.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.579+0000 7f3a618cf700 1 Processor -- start 2024-05-03T20:32:47.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.580+0000 7f3a618cf700 1 -- start start 2024-05-03T20:32:47.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.580+0000 7f3a618cf700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a5c103750 0x7f3a5c19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:47.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.580+0000 7f3a618cf700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a5c104100 0x7f3a5c19fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:47.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.580+0000 7f3a618cf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a5c108850 0x7f3a5c1999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:47.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.580+0000 7f3a618cf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c113710 con 0x7f3a5c108850 2024-05-03T20:32:47.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.580+0000 7f3a618cf700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c113590 con 0x7f3a5c103750 2024-05-03T20:32:47.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.580+0000 7f3a618cf700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c113890 con 0x7f3a5c104100 2024-05-03T20:32:47.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.580+0000 7f3a5b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a5c108850 0x7f3a5c1999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:47.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.580+0000 7f3a5b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a5c108850 0x7f3a5c1999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54024/0 (socket says 172.21.15.181:54024) 2024-05-03T20:32:47.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.580+0000 7f3a5b7fe700 1 -- 172.21.15.181:0/4144320125 learned_addr learned my addr 172.21.15.181:0/4144320125 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:32:47.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.580+0000 7f3a5a7fc700 1 --2- 172.21.15.181:0/4144320125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a5c104100 0x7f3a5c19fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:47.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.580+0000 7f3a5affd700 1 --2- 172.21.15.181:0/4144320125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a5c103750 0x7f3a5c19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:47.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.580+0000 7f3a5b7fe700 1 -- 172.21.15.181:0/4144320125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a5c104100 msgr2=0x7f3a5c19fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:47.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.580+0000 7f3a5b7fe700 1 --2- 172.21.15.181:0/4144320125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a5c104100 0x7f3a5c19fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:47.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.580+0000 7f3a5b7fe700 1 -- 172.21.15.181:0/4144320125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a5c103750 msgr2=0x7f3a5c19f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:47.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.580+0000 7f3a5b7fe700 1 --2- 172.21.15.181:0/4144320125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a5c103750 0x7f3a5c19f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:47.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.580+0000 7f3a5b7fe700 1 -- 172.21.15.181:0/4144320125 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5c19a130 con 0x7f3a5c108850 2024-05-03T20:32:47.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.581+0000 7f3a5b7fe700 1 --2- 172.21.15.181:0/4144320125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a5c108850 0x7f3a5c1999d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400b970 tx=0x7f3a5400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:32:47.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.581+0000 7f3a608cd700 1 -- 172.21.15.181:0/4144320125 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5400d860 con 0x7f3a5c108850 2024-05-03T20:32:47.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.581+0000 7f3a618cf700 1 -- 172.21.15.181:0/4144320125 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a5c19a420 con 0x7f3a5c108850 2024-05-03T20:32:47.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.581+0000 7f3a608cd700 1 -- 172.21.15.181:0/4144320125 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a5400e430 con 0x7f3a5c108850 2024-05-03T20:32:47.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.581+0000 7f3a608cd700 1 -- 172.21.15.181:0/4144320125 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54017660 con 0x7f3a5c108850 2024-05-03T20:32:47.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.581+0000 7f3a618cf700 1 -- 172.21.15.181:0/4144320125 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a5c1a66e0 con 0x7f3a5c108850 2024-05-03T20:32:47.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.582+0000 7f3a618cf700 1 -- 172.21.15.181:0/4144320125 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a5c06a190 con 0x7f3a5c108850 2024-05-03T20:32:47.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.584+0000 7f3a608cd700 1 -- 172.21.15.181:0/4144320125 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f3a5400da00 con 0x7f3a5c108850 2024-05-03T20:32:47.584 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.584+0000 7f3a608cd700 1 --2- 172.21.15.181:0/4144320125 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3a440b2880 0x7f3a440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:47.584 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.584+0000 7f3a5affd700 1 --2- 172.21.15.181:0/4144320125 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3a440b2880 0x7f3a440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:47.584 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.584+0000 7f3a608cd700 1 -- 172.21.15.181:0/4144320125 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f3a540e1b30 con 0x7f3a5c108850 2024-05-03T20:32:47.584 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.585+0000 7f3a5affd700 1 --2- 172.21.15.181:0/4144320125 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3a440b2880 0x7f3a440b4d40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c19ab10 tx=0x7f3a4c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:32:47.587 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.588+0000 7f3a608cd700 1 -- 172.21.15.181:0/4144320125 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3a54053d70 con 0x7f3a5c108850 2024-05-03T20:32:47.789 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.789+0000 7f3a618cf700 1 -- 172.21.15.181:0/4144320125 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3a5c19af00 con 0x7f3a5c108850 2024-05-03T20:32:47.789 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.790+0000 7f3a608cd700 1 -- 172.21.15.181:0/4144320125 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7f3a54013030 con 0x7f3a5c108850 2024-05-03T20:32:47.789 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2024-05-03T20:32:47.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.792+0000 7f3a618cf700 1 -- 172.21.15.181:0/4144320125 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3a440b2880 msgr2=0x7f3a440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:47.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.792+0000 7f3a618cf700 1 --2- 172.21.15.181:0/4144320125 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3a440b2880 0x7f3a440b4d40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c19ab10 tx=0x7f3a4c009190 comp rx=0 tx=0).stop 2024-05-03T20:32:47.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.792+0000 7f3a618cf700 1 -- 172.21.15.181:0/4144320125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a5c108850 msgr2=0x7f3a5c1999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:47.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.792+0000 7f3a618cf700 1 --2- 172.21.15.181:0/4144320125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a5c108850 0x7f3a5c1999d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400b970 tx=0x7f3a5400bc80 comp rx=0 tx=0).stop 2024-05-03T20:32:47.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.792+0000 7f3a618cf700 1 -- 172.21.15.181:0/4144320125 shutdown_connections 2024-05-03T20:32:47.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.792+0000 7f3a618cf700 1 --2- 172.21.15.181:0/4144320125 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3a440b2880 0x7f3a440b4d40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:47.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.792+0000 7f3a618cf700 1 --2- 172.21.15.181:0/4144320125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a5c108850 0x7f3a5c1999d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:47.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.792+0000 7f3a618cf700 1 --2- 172.21.15.181:0/4144320125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a5c103750 0x7f3a5c19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:47.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.792+0000 7f3a618cf700 1 --2- 172.21.15.181:0/4144320125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a5c104100 0x7f3a5c19fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:47.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.792+0000 7f3a618cf700 1 -- 172.21.15.181:0/4144320125 >> 172.21.15.181:0/4144320125 conn(0x7f3a5c07a600 msgr2=0x7f3a5c101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:32:47.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.792+0000 7f3a618cf700 1 -- 172.21.15.181:0/4144320125 shutdown_connections 2024-05-03T20:32:47.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:47.792+0000 7f3a618cf700 1 -- 172.21.15.181:0/4144320125 wait complete. 2024-05-03T20:32:48.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:47 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/4144320125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-03T20:32:48.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:47 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/4144320125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-03T20:32:48.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:47 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/4144320125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-03T20:32:48.520 INFO:tasks.cephadm:Adding mds.a on smithi181 2024-05-03T20:32:48.520 INFO:tasks.cephadm:Adding mds.d on smithi181 2024-05-03T20:32:48.520 INFO:tasks.cephadm:Adding mds.g on smithi181 2024-05-03T20:32:48.520 INFO:tasks.cephadm:Adding mds.j on smithi181 2024-05-03T20:32:48.520 INFO:tasks.cephadm:Adding mds.b on smithi196 2024-05-03T20:32:48.520 INFO:tasks.cephadm:Adding mds.e on smithi196 2024-05-03T20:32:48.521 INFO:tasks.cephadm:Adding mds.h on smithi196 2024-05-03T20:32:48.521 INFO:tasks.cephadm:Adding mds.k on smithi196 2024-05-03T20:32:48.521 INFO:tasks.cephadm:Adding mds.c on smithi204 2024-05-03T20:32:48.521 INFO:tasks.cephadm:Adding mds.f on smithi204 2024-05-03T20:32:48.521 INFO:tasks.cephadm:Adding mds.i on smithi204 2024-05-03T20:32:48.521 INFO:tasks.cephadm:Adding mds.l on smithi204 2024-05-03T20:32:48.521 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph orch apply mds all '12;smithi181=a;smithi181=d;smithi181=g;smithi181=j;smithi196=b;smithi196=e;smithi196=h;smithi196=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l' 2024-05-03T20:32:48.774 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T20:32:49.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:48 smithi204 ceph-mon[33595]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:49.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:48 smithi181 ceph-mon[29801]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:49.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:48 smithi196 ceph-mon[32858]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:50.599 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.599+0000 7f7c03b2f700 1 -- 172.21.15.204:0/2333278168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bfc102410 msgr2=0x7f7bfc102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:50.599 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.599+0000 7f7c03b2f700 1 --2- 172.21.15.204:0/2333278168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bfc102410 0x7f7bfc102830 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8009990 tx=0x7f7bf8009ca0 comp rx=0 tx=0).stop 2024-05-03T20:32:50.599 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.599+0000 7f7c03b2f700 1 -- 172.21.15.204:0/2333278168 shutdown_connections 2024-05-03T20:32:50.599 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.599+0000 7f7c03b2f700 1 --2- 172.21.15.204:0/2333278168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bfc103fb0 0x7f7bfc10a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:50.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.599+0000 7f7c03b2f700 1 --2- 172.21.15.204:0/2333278168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bfc102410 0x7f7bfc102830 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:50.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.599+0000 7f7c03b2f700 1 --2- 172.21.15.204:0/2333278168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bfc103610 0x7f7bfc103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:50.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.599+0000 7f7c03b2f700 1 -- 172.21.15.204:0/2333278168 >> 172.21.15.204:0/2333278168 conn(0x7f7bfc0fd9d0 msgr2=0x7f7bfc0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:32:50.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.599+0000 7f7c03b2f700 1 -- 172.21.15.204:0/2333278168 shutdown_connections 2024-05-03T20:32:50.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.599+0000 7f7c03b2f700 1 -- 172.21.15.204:0/2333278168 wait complete. 2024-05-03T20:32:50.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.599+0000 7f7c03b2f700 1 Processor -- start 2024-05-03T20:32:50.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.599+0000 7f7c03b2f700 1 -- start start 2024-05-03T20:32:50.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.600+0000 7f7c03b2f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bfc102410 0x7f7bfc19ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:50.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.600+0000 7f7c03b2f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bfc103610 0x7f7bfc19bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:50.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.600+0000 7f7c03b2f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bfc103fb0 0x7f7bfc1a2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:50.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.600+0000 7f7c03b2f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7bfc10df00 con 0x7f7bfc102410 2024-05-03T20:32:50.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.600+0000 7f7c03b2f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7bfc10dd80 con 0x7f7bfc103610 2024-05-03T20:32:50.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.600+0000 7f7c03b2f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7bfc10e080 con 0x7f7bfc103fb0 2024-05-03T20:32:50.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.600+0000 7f7c020cc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bfc103fb0 0x7f7bfc1a2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:50.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.600+0000 7f7c020cc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bfc103fb0 0x7f7bfc1a2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:52990/0 (socket says 172.21.15.204:52990) 2024-05-03T20:32:50.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.600+0000 7f7c020cc700 1 -- 172.21.15.204:0/1316492547 learned_addr learned my addr 172.21.15.204:0/1316492547 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-03T20:32:50.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.600+0000 7f7c018cb700 1 --2- 172.21.15.204:0/1316492547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bfc102410 0x7f7bfc19ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:50.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.600+0000 7f7c010ca700 1 --2- 172.21.15.204:0/1316492547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bfc103610 0x7f7bfc19bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:50.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.600+0000 7f7c020cc700 1 -- 172.21.15.204:0/1316492547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bfc103610 msgr2=0x7f7bfc19bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:50.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.600+0000 7f7c020cc700 1 --2- 172.21.15.204:0/1316492547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bfc103610 0x7f7bfc19bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:50.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.600+0000 7f7c020cc700 1 -- 172.21.15.204:0/1316492547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bfc102410 msgr2=0x7f7bfc19ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:50.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.600+0000 7f7c020cc700 1 --2- 172.21.15.204:0/1316492547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bfc102410 0x7f7bfc19ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:50.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.600+0000 7f7c020cc700 1 -- 172.21.15.204:0/1316492547 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bfc1a3630 con 0x7f7bfc103fb0 2024-05-03T20:32:50.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.600+0000 7f7c018cb700 1 --2- 172.21.15.204:0/1316492547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bfc102410 0x7f7bfc19ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:32:50.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.600+0000 7f7c020cc700 1 --2- 172.21.15.204:0/1316492547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bfc103fb0 0x7f7bfc1a2f90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400add0 tx=0x7f7bf400c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:32:50.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.600+0000 7f7bf2ffd700 1 -- 172.21.15.204:0/1316492547 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4017020 con 0x7f7bfc103fb0 2024-05-03T20:32:50.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.600+0000 7f7bf2ffd700 1 -- 172.21.15.204:0/1316492547 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bf400cdf0 con 0x7f7bfc103fb0 2024-05-03T20:32:50.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.600+0000 7f7c03b2f700 1 -- 172.21.15.204:0/1316492547 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bfc1a3920 con 0x7f7bfc103fb0 2024-05-03T20:32:50.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.600+0000 7f7bf2ffd700 1 -- 172.21.15.204:0/1316492547 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4005460 con 0x7f7bfc103fb0 2024-05-03T20:32:50.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.601+0000 7f7c03b2f700 1 -- 172.21.15.204:0/1316492547 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bfc1a3e60 con 0x7f7bfc103fb0 2024-05-03T20:32:50.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.602+0000 7f7c03b2f700 1 -- 172.21.15.204:0/1316492547 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bfc06a1c0 con 0x7f7bfc103fb0 2024-05-03T20:32:50.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.603+0000 7f7bf2ffd700 1 -- 172.21.15.204:0/1316492547 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f7bf4004ad0 con 0x7f7bfc103fb0 2024-05-03T20:32:50.604 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.603+0000 7f7bf2ffd700 1 --2- 172.21.15.204:0/1316492547 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f7be80b2900 0x7f7be80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:50.604 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.603+0000 7f7bf2ffd700 1 -- 172.21.15.204:0/1316492547 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f7bf40e1590 con 0x7f7bfc103fb0 2024-05-03T20:32:50.604 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.603+0000 7f7c018cb700 1 --2- 172.21.15.204:0/1316492547 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f7be80b2900 0x7f7be80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:50.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.604+0000 7f7c018cb700 1 --2- 172.21.15.204:0/1316492547 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f7be80b2900 0x7f7be80b4dc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800b5c0 tx=0x7f7bf8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:32:50.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.608+0000 7f7bf2ffd700 1 -- 172.21.15.204:0/1316492547 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7bfc06a1c0 con 0x7f7bfc103fb0 2024-05-03T20:32:50.817 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.818+0000 7f7c03b2f700 1 -- 172.21.15.204:0/1316492547 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi181=a;smithi181=d;smithi181=g;smithi181=j;smithi196=b;smithi196=e;smithi196=h;smithi196=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}) -- 0x7f7bfc06c450 con 0x7f7be80b2900 2024-05-03T20:32:50.825 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.827+0000 7f7bf2ffd700 1 -- 172.21.15.204:0/1316492547 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f7bfc06c450 con 0x7f7be80b2900 2024-05-03T20:32:50.826 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mds.all update... 2024-05-03T20:32:50.828 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.829+0000 7f7c03b2f700 1 -- 172.21.15.204:0/1316492547 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f7be80b2900 msgr2=0x7f7be80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:50.828 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.829+0000 7f7c03b2f700 1 --2- 172.21.15.204:0/1316492547 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f7be80b2900 0x7f7be80b4dc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800b5c0 tx=0x7f7bf8005fb0 comp rx=0 tx=0).stop 2024-05-03T20:32:50.829 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.829+0000 7f7c03b2f700 1 -- 172.21.15.204:0/1316492547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bfc103fb0 msgr2=0x7f7bfc1a2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:50.829 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.829+0000 7f7c03b2f700 1 --2- 172.21.15.204:0/1316492547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bfc103fb0 0x7f7bfc1a2f90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400add0 tx=0x7f7bf400c330 comp rx=0 tx=0).stop 2024-05-03T20:32:50.829 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.829+0000 7f7c03b2f700 1 -- 172.21.15.204:0/1316492547 shutdown_connections 2024-05-03T20:32:50.829 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.829+0000 7f7c03b2f700 1 --2- 172.21.15.204:0/1316492547 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f7be80b2900 0x7f7be80b4dc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:50.829 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.829+0000 7f7c03b2f700 1 --2- 172.21.15.204:0/1316492547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bfc103fb0 0x7f7bfc1a2f90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:50.829 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.829+0000 7f7c03b2f700 1 --2- 172.21.15.204:0/1316492547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bfc102410 0x7f7bfc19ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:50.829 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.829+0000 7f7c03b2f700 1 --2- 172.21.15.204:0/1316492547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bfc103610 0x7f7bfc19bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:50.829 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.829+0000 7f7c03b2f700 1 -- 172.21.15.204:0/1316492547 >> 172.21.15.204:0/1316492547 conn(0x7f7bfc0fd9d0 msgr2=0x7f7bfc06bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:32:50.829 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.829+0000 7f7c03b2f700 1 -- 172.21.15.204:0/1316492547 shutdown_connections 2024-05-03T20:32:50.829 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-03T20:32:50.829+0000 7f7c03b2f700 1 -- 172.21.15.204:0/1316492547 wait complete. 2024-05-03T20:32:50.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:50 smithi181 ceph-mon[29801]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:50.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:50 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:50.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:50 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:32:51.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:50 smithi204 ceph-mon[33595]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:51.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:50 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:51.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:50 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:32:51.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:50 smithi196 ceph-mon[32858]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:51.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:50 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:51.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:50 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:32:51.445 DEBUG:teuthology.orchestra.run.smithi181:mds.a> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.a.service 2024-05-03T20:32:51.448 DEBUG:teuthology.orchestra.run.smithi181:mds.d> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.d.service 2024-05-03T20:32:51.492 DEBUG:teuthology.orchestra.run.smithi181:mds.g> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.g.service 2024-05-03T20:32:51.535 DEBUG:teuthology.orchestra.run.smithi181:mds.j> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.j.service 2024-05-03T20:32:51.577 DEBUG:teuthology.orchestra.run.smithi196:mds.b> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.b.service 2024-05-03T20:32:51.580 DEBUG:teuthology.orchestra.run.smithi196:mds.e> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.e.service 2024-05-03T20:32:51.623 DEBUG:teuthology.orchestra.run.smithi196:mds.h> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.h.service 2024-05-03T20:32:51.666 DEBUG:teuthology.orchestra.run.smithi196:mds.k> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.k.service 2024-05-03T20:32:51.710 DEBUG:teuthology.orchestra.run.smithi204:mds.c> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.c.service 2024-05-03T20:32:51.712 DEBUG:teuthology.orchestra.run.smithi204:mds.f> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.f.service 2024-05-03T20:32:51.755 DEBUG:teuthology.orchestra.run.smithi204:mds.i> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.i.service 2024-05-03T20:32:51.798 DEBUG:teuthology.orchestra.run.smithi204:mds.l> sudo journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.l.service 2024-05-03T20:32:51.840 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-03T20:32:51.841 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T20:32:52.079 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:32:52.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:51 smithi204 ceph-mon[33595]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi181=a;smithi181=d;smithi181=g;smithi181=j;smithi196=b;smithi196=e;smithi196=h;smithi196=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:32:52.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:51 smithi204 ceph-mon[33595]: Saving service mds.all spec with placement smithi181=a;smithi181=d;smithi181=g;smithi181=j;smithi196=b;smithi196=e;smithi196=h;smithi196=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-05-03T20:32:52.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:51 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:52.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:51 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:32:52.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:51 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:52.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:51 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:32:52.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:51 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:32:52.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:51 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:52.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:51 smithi204 ceph-mon[33595]: Deploying daemon mds.l on smithi204 2024-05-03T20:32:52.202 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:51 smithi181 ceph-mon[29801]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi181=a;smithi181=d;smithi181=g;smithi181=j;smithi196=b;smithi196=e;smithi196=h;smithi196=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:32:52.202 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:51 smithi181 ceph-mon[29801]: Saving service mds.all spec with placement smithi181=a;smithi181=d;smithi181=g;smithi181=j;smithi196=b;smithi196=e;smithi196=h;smithi196=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-05-03T20:32:52.202 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:51 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:52.202 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:51 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:32:52.203 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:51 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:52.203 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:51 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:32:52.203 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:51 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:32:52.203 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:51 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:52.203 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:51 smithi181 ceph-mon[29801]: Deploying daemon mds.l on smithi204 2024-05-03T20:32:52.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:51 smithi196 ceph-mon[32858]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi181=a;smithi181=d;smithi181=g;smithi181=j;smithi196=b;smithi196=e;smithi196=h;smithi196=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:32:52.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:51 smithi196 ceph-mon[32858]: Saving service mds.all spec with placement smithi181=a;smithi181=d;smithi181=g;smithi181=j;smithi196=b;smithi196=e;smithi196=h;smithi196=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-05-03T20:32:52.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:51 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:52.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:51 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:32:52.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:51 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:52.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:51 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:32:52.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:51 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:32:52.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:51 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:52.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:51 smithi196 ceph-mon[32858]: Deploying daemon mds.l on smithi204 2024-05-03T20:32:53.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.756+0000 7fb1845c9700 1 -- 172.21.15.181:0/1395832374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb17c1076e0 msgr2=0x7fb17c10f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:53.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.756+0000 7fb1845c9700 1 --2- 172.21.15.181:0/1395832374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb17c1076e0 0x7fb17c10f2b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb16c0099c0 tx=0x7fb16c009cd0 comp rx=0 tx=0).stop 2024-05-03T20:32:53.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.757+0000 7fb1845c9700 1 -- 172.21.15.181:0/1395832374 shutdown_connections 2024-05-03T20:32:53.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.757+0000 7fb1845c9700 1 --2- 172.21.15.181:0/1395832374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb17c1076e0 0x7fb17c10f2b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:53.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.757+0000 7fb1845c9700 1 --2- 172.21.15.181:0/1395832374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb17c102690 0x7fb17c102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:53.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.757+0000 7fb1845c9700 1 --2- 172.21.15.181:0/1395832374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb17c103040 0x7fb17c107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:53.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.757+0000 7fb1845c9700 1 -- 172.21.15.181:0/1395832374 >> 172.21.15.181:0/1395832374 conn(0x7fb17c0783f0 msgr2=0x7fb17c078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:32:53.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.757+0000 7fb1845c9700 1 -- 172.21.15.181:0/1395832374 shutdown_connections 2024-05-03T20:32:53.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.757+0000 7fb1845c9700 1 -- 172.21.15.181:0/1395832374 wait complete. 2024-05-03T20:32:53.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.757+0000 7fb1845c9700 1 Processor -- start 2024-05-03T20:32:53.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.757+0000 7fb1845c9700 1 -- start start 2024-05-03T20:32:53.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.757+0000 7fb1845c9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb17c102690 0x7fb17c19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:53.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.757+0000 7fb1845c9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb17c103040 0x7fb17c19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:53.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.757+0000 7fb1845c9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb17c1076e0 0x7fb17c199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:53.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.757+0000 7fb1845c9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c1125a0 con 0x7fb17c103040 2024-05-03T20:32:53.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.758+0000 7fb1845c9700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c112420 con 0x7fb17c102690 2024-05-03T20:32:53.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.758+0000 7fb1845c9700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c112720 con 0x7fb17c1076e0 2024-05-03T20:32:53.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.758+0000 7fb181b64700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb17c103040 0x7fb17c19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:53.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.758+0000 7fb181b64700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb17c103040 0x7fb17c19fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54042/0 (socket says 172.21.15.181:54042) 2024-05-03T20:32:53.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.758+0000 7fb181b64700 1 -- 172.21.15.181:0/2904360596 learned_addr learned my addr 172.21.15.181:0/2904360596 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:32:53.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.758+0000 7fb182b66700 1 --2- 172.21.15.181:0/2904360596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb17c1076e0 0x7fb17c199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:53.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.758+0000 7fb182365700 1 --2- 172.21.15.181:0/2904360596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb17c102690 0x7fb17c19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:53.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.758+0000 7fb181b64700 1 -- 172.21.15.181:0/2904360596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb17c1076e0 msgr2=0x7fb17c199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:53.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.758+0000 7fb181b64700 1 --2- 172.21.15.181:0/2904360596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb17c1076e0 0x7fb17c199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:53.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.758+0000 7fb181b64700 1 -- 172.21.15.181:0/2904360596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb17c102690 msgr2=0x7fb17c19f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:53.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.758+0000 7fb181b64700 1 --2- 172.21.15.181:0/2904360596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb17c102690 0x7fb17c19f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:53.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.758+0000 7fb181b64700 1 -- 172.21.15.181:0/2904360596 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb17c19a0c0 con 0x7fb17c103040 2024-05-03T20:32:53.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.758+0000 7fb181b64700 1 --2- 172.21.15.181:0/2904360596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb17c103040 0x7fb17c19fce0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb17000bd40 tx=0x7fb170009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:32:53.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.758+0000 7fb17b7fe700 1 -- 172.21.15.181:0/2904360596 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb170011050 con 0x7fb17c103040 2024-05-03T20:32:53.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.758+0000 7fb17b7fe700 1 -- 172.21.15.181:0/2904360596 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1700106e0 con 0x7fb17c103040 2024-05-03T20:32:53.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.758+0000 7fb1845c9700 1 -- 172.21.15.181:0/2904360596 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb17c19a3b0 con 0x7fb17c103040 2024-05-03T20:32:53.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.758+0000 7fb17b7fe700 1 -- 172.21.15.181:0/2904360596 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17001a600 con 0x7fb17c103040 2024-05-03T20:32:53.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.758+0000 7fb1845c9700 1 -- 172.21.15.181:0/2904360596 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb17c103fd0 con 0x7fb17c103040 2024-05-03T20:32:53.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.759+0000 7fb1845c9700 1 -- 172.21.15.181:0/2904360596 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb17c06a190 con 0x7fb17c103040 2024-05-03T20:32:53.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.760+0000 7fb17b7fe700 1 -- 172.21.15.181:0/2904360596 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fb170010890 con 0x7fb17c103040 2024-05-03T20:32:53.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.761+0000 7fb17b7fe700 1 --2- 172.21.15.181:0/2904360596 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb1640b2900 0x7fb1640b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:53.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.761+0000 7fb182365700 1 --2- 172.21.15.181:0/2904360596 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb1640b2900 0x7fb1640b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:53.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.761+0000 7fb17b7fe700 1 -- 172.21.15.181:0/2904360596 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fb1700e5a10 con 0x7fb17c103040 2024-05-03T20:32:53.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.761+0000 7fb182365700 1 --2- 172.21.15.181:0/2904360596 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb1640b2900 0x7fb1640b4dc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb17c19aaa0 tx=0x7fb17400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:32:53.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:53.766+0000 7fb17b7fe700 1 -- 172.21.15.181:0/2904360596 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb170016070 con 0x7fb17c103040 2024-05-03T20:32:53.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:53 smithi204 ceph-mon[33595]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:53.863 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 03 20:32:53 smithi204 systemd[1]: Starting Ceph mds.l for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:32:53.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:53 smithi181 ceph-mon[29801]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:54.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:54.021+0000 7fb1845c9700 1 -- 172.21.15.181:0/2904360596 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb17c02e510 con 0x7fb17c103040 2024-05-03T20:32:54.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:54.022+0000 7fb17b7fe700 1 -- 172.21.15.181:0/2904360596 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fb1700adbd0 con 0x7fb17c103040 2024-05-03T20:32:54.021 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:32:54.022 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-03T20:32:54.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:54.024+0000 7fb1845c9700 1 -- 172.21.15.181:0/2904360596 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb1640b2900 msgr2=0x7fb1640b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:54.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:54.024+0000 7fb1845c9700 1 --2- 172.21.15.181:0/2904360596 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb1640b2900 0x7fb1640b4dc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb17c19aaa0 tx=0x7fb17400b040 comp rx=0 tx=0).stop 2024-05-03T20:32:54.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:54.024+0000 7fb1845c9700 1 -- 172.21.15.181:0/2904360596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb17c103040 msgr2=0x7fb17c19fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:54.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:54.024+0000 7fb1845c9700 1 --2- 172.21.15.181:0/2904360596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb17c103040 0x7fb17c19fce0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb17000bd40 tx=0x7fb170009520 comp rx=0 tx=0).stop 2024-05-03T20:32:54.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:54.024+0000 7fb1845c9700 1 -- 172.21.15.181:0/2904360596 shutdown_connections 2024-05-03T20:32:54.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:54.024+0000 7fb1845c9700 1 --2- 172.21.15.181:0/2904360596 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb1640b2900 0x7fb1640b4dc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:54.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:54.024+0000 7fb1845c9700 1 --2- 172.21.15.181:0/2904360596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb17c1076e0 0x7fb17c199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:54.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:54.024+0000 7fb1845c9700 1 --2- 172.21.15.181:0/2904360596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb17c102690 0x7fb17c19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:54.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:54.024+0000 7fb1845c9700 1 --2- 172.21.15.181:0/2904360596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb17c103040 0x7fb17c19fce0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:54.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:54.024+0000 7fb1845c9700 1 -- 172.21.15.181:0/2904360596 >> 172.21.15.181:0/2904360596 conn(0x7fb17c0783f0 msgr2=0x7fb17c06be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:32:54.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:54.024+0000 7fb1845c9700 1 -- 172.21.15.181:0/2904360596 shutdown_connections 2024-05-03T20:32:54.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:54.024+0000 7fb1845c9700 1 -- 172.21.15.181:0/2904360596 wait complete. 2024-05-03T20:32:54.025 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 1 2024-05-03T20:32:54.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:53 smithi196 ceph-mon[32858]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:54.181 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 03 20:32:53 smithi204 podman[63616]: 2024-05-03 20:32:53.791543897 +0000 UTC m=+0.018742483 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T20:32:54.182 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 03 20:32:53 smithi204 podman[63616]: 2024-05-03 20:32:53.897006038 +0000 UTC m=+0.124204595 container create 271a1961e50251175d5ef9bcea4e0e207fe041be3dfa74612cedf5d95173c8c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-l, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-03T20:32:54.182 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 03 20:32:54 smithi204 podman[63616]: 2024-05-03 20:32:54.107989518 +0000 UTC m=+0.335188088 container init 271a1961e50251175d5ef9bcea4e0e207fe041be3dfa74612cedf5d95173c8c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-l, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True) 2024-05-03T20:32:54.182 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 03 20:32:54 smithi204 podman[63616]: 2024-05-03 20:32:54.110700533 +0000 UTC m=+0.337899099 container start 271a1961e50251175d5ef9bcea4e0e207fe041be3dfa74612cedf5d95173c8c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-l, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T20:32:54.182 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 03 20:32:54 smithi204 ceph-mds[63635]: -- 172.21.15.204:0/2672009274 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5641f6247a40 con 0x5641f621ac00 2024-05-03T20:32:54.182 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 03 20:32:54 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-l[63631]: starting mds.l at 2024-05-03T20:32:54.182 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 03 20:32:54 smithi204 bash[63616]: 271a1961e50251175d5ef9bcea4e0e207fe041be3dfa74612cedf5d95173c8c5 2024-05-03T20:32:54.613 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 03 20:32:54 smithi204 systemd[1]: Started Ceph mds.l for 1cd058ee-098b-11ef-bc97-c7b262605968. 2024-05-03T20:32:54.677 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:54 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/2904360596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:32:54.677 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:54 smithi181 ceph-mon[29801]: mds.? [v2:172.21.15.204:6832/2409628823,v1:172.21.15.204:6833/2409628823] up:boot 2024-05-03T20:32:54.677 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:54 smithi181 ceph-mon[29801]: fsmap 1 up:standby 2024-05-03T20:32:54.678 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:54 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T20:32:54.678 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:54 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:54.678 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:54 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:54.678 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:54 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:54.678 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:54 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:32:54.678 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:54 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:32:54.678 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:54 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:54.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:54 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/2904360596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:32:54.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:54 smithi196 ceph-mon[32858]: mds.? [v2:172.21.15.204:6832/2409628823,v1:172.21.15.204:6833/2409628823] up:boot 2024-05-03T20:32:54.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:54 smithi196 ceph-mon[32858]: fsmap 1 up:standby 2024-05-03T20:32:54.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:54 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T20:32:54.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:54 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:54.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:54 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:54.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:54 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:54.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:54 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:32:54.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:54 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:32:54.922 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:54 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:55.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:54 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/2904360596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:32:55.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:54 smithi204 ceph-mon[33595]: mds.? [v2:172.21.15.204:6832/2409628823,v1:172.21.15.204:6833/2409628823] up:boot 2024-05-03T20:32:55.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:54 smithi204 ceph-mon[33595]: fsmap 1 up:standby 2024-05-03T20:32:55.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:54 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T20:32:55.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:54 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:55.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:54 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:55.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:54 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:55.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:54 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:32:55.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:54 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:32:55.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:54 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:55.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:55 smithi181 ceph-mon[29801]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:55.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:55 smithi181 ceph-mon[29801]: Deploying daemon mds.k on smithi196 2024-05-03T20:32:56.028 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:55 smithi196 ceph-mon[32858]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:56.028 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:55 smithi196 ceph-mon[32858]: Deploying daemon mds.k on smithi196 2024-05-03T20:32:56.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:55 smithi204 ceph-mon[33595]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:56.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:55 smithi204 ceph-mon[33595]: Deploying daemon mds.k on smithi196 2024-05-03T20:32:56.356 INFO:journalctl@ceph.mds.k.smithi196.stdout:May 03 20:32:56 smithi196 systemd[1]: Starting Ceph mds.k for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:32:56.679 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T20:32:56.723 INFO:journalctl@ceph.mds.k.smithi196.stdout:May 03 20:32:56 smithi196 podman[62585]: 2024-05-03 20:32:56.360236346 +0000 UTC m=+0.017821772 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T20:32:56.723 INFO:journalctl@ceph.mds.k.smithi196.stdout:May 03 20:32:56 smithi196 podman[62585]: 2024-05-03 20:32:56.47276309 +0000 UTC m=+0.130348516 container create dcb17aadb68d2050a88c5370bb150af6f345cf8ebb100d8deea7c2acb6c80c52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-k, ceph=True, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0) 2024-05-03T20:32:56.723 INFO:journalctl@ceph.mds.k.smithi196.stdout:May 03 20:32:56 smithi196 podman[62585]: 2024-05-03 20:32:56.725010347 +0000 UTC m=+0.382595774 container init dcb17aadb68d2050a88c5370bb150af6f345cf8ebb100d8deea7c2acb6c80c52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T20:32:56.931 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:32:57.106 INFO:journalctl@ceph.mds.k.smithi196.stdout:May 03 20:32:56 smithi196 podman[62585]: 2024-05-03 20:32:56.728197109 +0000 UTC m=+0.385782544 container start dcb17aadb68d2050a88c5370bb150af6f345cf8ebb100d8deea7c2acb6c80c52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-k, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2) 2024-05-03T20:32:57.106 INFO:journalctl@ceph.mds.k.smithi196.stdout:May 03 20:32:56 smithi196 ceph-mds[62605]: -- 172.21.15.196:0/1244222298 <== mon.2 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56438c95ba40 con 0x56438c92e800 2024-05-03T20:32:57.107 INFO:journalctl@ceph.mds.k.smithi196.stdout:May 03 20:32:56 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-k[62601]: starting mds.k at 2024-05-03T20:32:57.107 INFO:journalctl@ceph.mds.k.smithi196.stdout:May 03 20:32:56 smithi196 bash[62585]: dcb17aadb68d2050a88c5370bb150af6f345cf8ebb100d8deea7c2acb6c80c52 2024-05-03T20:32:57.107 INFO:journalctl@ceph.mds.k.smithi196.stdout:May 03 20:32:56 smithi196 systemd[1]: Started Ceph mds.k for 1cd058ee-098b-11ef-bc97-c7b262605968. 2024-05-03T20:32:57.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:57 smithi181 ceph-mon[29801]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:57.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:57 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:57.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:57 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:57.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:57 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:57.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:57 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:32:57.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:57 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:32:57.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:57 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:58.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:57 smithi196 ceph-mon[32858]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:58.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:57 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:58.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:57 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:58.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:57 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:58.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:57 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:32:58.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:57 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:32:58.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:57 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:58.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:57 smithi204 ceph-mon[33595]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:58.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:57 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:58.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:57 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:58.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:57 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:32:58.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:57 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:32:58.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:57 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:32:58.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:57 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:32:58.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.492+0000 7f6c53d4d700 1 -- 172.21.15.181:0/3615247313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c4c103540 msgr2=0x7f6c4c1039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:58.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.492+0000 7f6c53d4d700 1 --2- 172.21.15.181:0/3615247313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c4c103540 0x7f6c4c1039c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0099c0 tx=0x7f6c3c009cd0 comp rx=0 tx=0).stop 2024-05-03T20:32:58.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.492+0000 7f6c53d4d700 1 -- 172.21.15.181:0/3615247313 shutdown_connections 2024-05-03T20:32:58.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.492+0000 7f6c53d4d700 1 --2- 172.21.15.181:0/3615247313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c4c103f00 0x7f6c4c1103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:58.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.492+0000 7f6c53d4d700 1 --2- 172.21.15.181:0/3615247313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c4c1095e0 0x7f6c4c1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:58.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.492+0000 7f6c53d4d700 1 --2- 172.21.15.181:0/3615247313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c4c103540 0x7f6c4c1039c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:58.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.492+0000 7f6c53d4d700 1 -- 172.21.15.181:0/3615247313 >> 172.21.15.181:0/3615247313 conn(0x7f6c4c07a650 msgr2=0x7f6c4c07aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:32:58.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.492+0000 7f6c53d4d700 1 -- 172.21.15.181:0/3615247313 shutdown_connections 2024-05-03T20:32:58.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.492+0000 7f6c53d4d700 1 -- 172.21.15.181:0/3615247313 wait complete. 2024-05-03T20:32:58.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.493+0000 7f6c53d4d700 1 Processor -- start 2024-05-03T20:32:58.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.493+0000 7f6c53d4d700 1 -- start start 2024-05-03T20:32:58.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.493+0000 7f6c53d4d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c4c103540 0x7f6c4c0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:58.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.493+0000 7f6c53d4d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c4c103f00 0x7f6c4c075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:58.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.493+0000 7f6c53d4d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c4c1095e0 0x7f6c4c076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:58.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.493+0000 7f6c53d4d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c113740 con 0x7f6c4c103f00 2024-05-03T20:32:58.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.493+0000 7f6c53d4d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1135c0 con 0x7f6c4c103540 2024-05-03T20:32:58.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.493+0000 7f6c53d4d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1138c0 con 0x7f6c4c1095e0 2024-05-03T20:32:58.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.493+0000 7f6c512e8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c4c103f00 0x7f6c4c075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:58.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.493+0000 7f6c51ae9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c4c103540 0x7f6c4c0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:58.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.493+0000 7f6c512e8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c4c103f00 0x7f6c4c075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52728/0 (socket says 172.21.15.181:52728) 2024-05-03T20:32:58.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.493+0000 7f6c512e8700 1 -- 172.21.15.181:0/473436843 learned_addr learned my addr 172.21.15.181:0/473436843 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:32:58.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.493+0000 7f6c522ea700 1 --2- 172.21.15.181:0/473436843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c4c1095e0 0x7f6c4c076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:58.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.493+0000 7f6c512e8700 1 -- 172.21.15.181:0/473436843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c4c1095e0 msgr2=0x7f6c4c076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:58.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.493+0000 7f6c512e8700 1 --2- 172.21.15.181:0/473436843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c4c1095e0 0x7f6c4c076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:58.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.494+0000 7f6c512e8700 1 -- 172.21.15.181:0/473436843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c4c103540 msgr2=0x7f6c4c0757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:58.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.494+0000 7f6c512e8700 1 --2- 172.21.15.181:0/473436843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c4c103540 0x7f6c4c0757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:58.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.494+0000 7f6c512e8700 1 -- 172.21.15.181:0/473436843 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4c1a8080 con 0x7f6c4c103f00 2024-05-03T20:32:58.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.494+0000 7f6c512e8700 1 --2- 172.21.15.181:0/473436843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c4c103f00 0x7f6c4c075ce0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0098e0 tx=0x7f6c3c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:32:58.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.494+0000 7f6c42ffd700 1 -- 172.21.15.181:0/473436843 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c027070 con 0x7f6c4c103f00 2024-05-03T20:32:58.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.494+0000 7f6c53d4d700 1 -- 172.21.15.181:0/473436843 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c4c1a8310 con 0x7f6c4c103f00 2024-05-03T20:32:58.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.494+0000 7f6c42ffd700 1 -- 172.21.15.181:0/473436843 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c3c003f20 con 0x7f6c4c103f00 2024-05-03T20:32:58.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.494+0000 7f6c42ffd700 1 -- 172.21.15.181:0/473436843 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c022ca0 con 0x7f6c4c103f00 2024-05-03T20:32:58.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.494+0000 7f6c53d4d700 1 -- 172.21.15.181:0/473436843 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c4c1a8760 con 0x7f6c4c103f00 2024-05-03T20:32:58.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.495+0000 7f6c53d4d700 1 -- 172.21.15.181:0/473436843 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c4c104c50 con 0x7f6c4c103f00 2024-05-03T20:32:58.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.496+0000 7f6c42ffd700 1 -- 172.21.15.181:0/473436843 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f6c3c02a070 con 0x7f6c4c103f00 2024-05-03T20:32:58.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.497+0000 7f6c42ffd700 1 --2- 172.21.15.181:0/473436843 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f6c380b2880 0x7f6c380b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:32:58.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.497+0000 7f6c51ae9700 1 --2- 172.21.15.181:0/473436843 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f6c380b2880 0x7f6c380b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:32:58.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.497+0000 7f6c42ffd700 1 -- 172.21.15.181:0/473436843 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f6c3c0e3450 con 0x7f6c4c103f00 2024-05-03T20:32:58.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.498+0000 7f6c51ae9700 1 --2- 172.21.15.181:0/473436843 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f6c380b2880 0x7f6c380b4d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6c48007b60 tx=0x7f6c480073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:32:58.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.501+0000 7f6c42ffd700 1 -- 172.21.15.181:0/473436843 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6c3c031020 con 0x7f6c4c103f00 2024-05-03T20:32:58.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.764+0000 7f6c53d4d700 1 -- 172.21.15.181:0/473436843 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c4c076cd0 con 0x7f6c4c103f00 2024-05-03T20:32:58.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.765+0000 7f6c42ffd700 1 -- 172.21.15.181:0/473436843 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f6c3c013070 con 0x7f6c4c103f00 2024-05-03T20:32:58.765 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:32:58.765 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3975688022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3975688022},{"type":"v1","addr":"172.21.15.196:6833","nonce":3975688022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2409628823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2409628823},{"type":"v1","addr":"172.21.15.204:6833","nonce":2409628823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-03T20:32:58.767 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.767+0000 7f6c53d4d700 1 -- 172.21.15.181:0/473436843 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f6c380b2880 msgr2=0x7f6c380b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:58.767 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.767+0000 7f6c53d4d700 1 --2- 172.21.15.181:0/473436843 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f6c380b2880 0x7f6c380b4d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6c48007b60 tx=0x7f6c480073d0 comp rx=0 tx=0).stop 2024-05-03T20:32:58.767 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.767+0000 7f6c53d4d700 1 -- 172.21.15.181:0/473436843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c4c103f00 msgr2=0x7f6c4c075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:32:58.767 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.767+0000 7f6c53d4d700 1 --2- 172.21.15.181:0/473436843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c4c103f00 0x7f6c4c075ce0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0098e0 tx=0x7f6c3c0057c0 comp rx=0 tx=0).stop 2024-05-03T20:32:58.767 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.767+0000 7f6c53d4d700 1 -- 172.21.15.181:0/473436843 shutdown_connections 2024-05-03T20:32:58.767 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.767+0000 7f6c53d4d700 1 --2- 172.21.15.181:0/473436843 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f6c380b2880 0x7f6c380b4d40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:58.767 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.767+0000 7f6c53d4d700 1 --2- 172.21.15.181:0/473436843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c4c1095e0 0x7f6c4c076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:58.767 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.767+0000 7f6c53d4d700 1 --2- 172.21.15.181:0/473436843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c4c103540 0x7f6c4c0757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:58.767 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.767+0000 7f6c53d4d700 1 --2- 172.21.15.181:0/473436843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c4c103f00 0x7f6c4c075ce0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:32:58.767 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.767+0000 7f6c53d4d700 1 -- 172.21.15.181:0/473436843 >> 172.21.15.181:0/473436843 conn(0x7f6c4c07a650 msgr2=0x7f6c4c102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:32:58.767 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.767+0000 7f6c53d4d700 1 -- 172.21.15.181:0/473436843 shutdown_connections 2024-05-03T20:32:58.767 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:32:58.767+0000 7f6c53d4d700 1 -- 172.21.15.181:0/473436843 wait complete. 2024-05-03T20:32:58.768 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 3 2024-05-03T20:32:58.924 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:58 smithi181 ceph-mon[29801]: Deploying daemon mds.j on smithi181 2024-05-03T20:32:58.924 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:58 smithi181 ceph-mon[29801]: mds.? [v2:172.21.15.196:6832/3975688022,v1:172.21.15.196:6833/3975688022] up:boot 2024-05-03T20:32:58.924 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:58 smithi181 ceph-mon[29801]: fsmap 2 up:standby 2024-05-03T20:32:58.924 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:58 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-03T20:32:59.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:58 smithi196 ceph-mon[32858]: Deploying daemon mds.j on smithi181 2024-05-03T20:32:59.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:58 smithi196 ceph-mon[32858]: mds.? [v2:172.21.15.196:6832/3975688022,v1:172.21.15.196:6833/3975688022] up:boot 2024-05-03T20:32:59.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:58 smithi196 ceph-mon[32858]: fsmap 2 up:standby 2024-05-03T20:32:59.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:58 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-03T20:32:59.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:58 smithi204 ceph-mon[33595]: Deploying daemon mds.j on smithi181 2024-05-03T20:32:59.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:58 smithi204 ceph-mon[33595]: mds.? [v2:172.21.15.196:6832/3975688022,v1:172.21.15.196:6833/3975688022] up:boot 2024-05-03T20:32:59.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:58 smithi204 ceph-mon[33595]: fsmap 2 up:standby 2024-05-03T20:32:59.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:58 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-03T20:32:59.927 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:59 smithi181 ceph-mon[29801]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:32:59.927 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:32:59 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/473436843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:33:00.066 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:59 smithi196 ceph-mon[32858]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:00.067 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:32:59 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/473436843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:33:00.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:59 smithi204 ceph-mon[33595]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:00.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:32:59 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/473436843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:33:00.213 INFO:journalctl@ceph.mds.j.smithi181.stdout:May 03 20:33:00 smithi181 systemd[1]: Starting Ceph mds.j for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:33:00.963 INFO:journalctl@ceph.mds.j.smithi181.stdout:May 03 20:33:00 smithi181 podman[70630]: 2024-05-03 20:33:00.568949941 +0000 UTC m=+0.018186618 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T20:33:01.031 INFO:journalctl@ceph.mds.j.smithi181.stdout:May 03 20:33:00 smithi181 podman[70630]: 2024-05-03 20:33:00.668372577 +0000 UTC m=+0.117609238 container create 37665f477d8460617beb522bc4914e37884ba9b73b0a7d81265eda0c1351c115 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True) 2024-05-03T20:33:01.033 INFO:journalctl@ceph.mds.j.smithi181.stdout:May 03 20:33:00 smithi181 podman[70630]: 2024-05-03 20:33:00.887548997 +0000 UTC m=+0.336785661 container init 37665f477d8460617beb522bc4914e37884ba9b73b0a7d81265eda0c1351c115 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-j, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS) 2024-05-03T20:33:01.033 INFO:journalctl@ceph.mds.j.smithi181.stdout:May 03 20:33:00 smithi181 podman[70630]: 2024-05-03 20:33:00.890077955 +0000 UTC m=+0.339314608 container start 37665f477d8460617beb522bc4914e37884ba9b73b0a7d81265eda0c1351c115 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-j, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-03T20:33:01.033 INFO:journalctl@ceph.mds.j.smithi181.stdout:May 03 20:33:00 smithi181 ceph-mds[70649]: -- 172.21.15.181:0/806265943 <== mon.0 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55758e12ba40 con 0x55758e0fec00 2024-05-03T20:33:01.033 INFO:journalctl@ceph.mds.j.smithi181.stdout:May 03 20:33:00 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-j[70645]: starting mds.j at 2024-05-03T20:33:01.463 INFO:journalctl@ceph.mds.j.smithi181.stdout:May 03 20:33:00 smithi181 bash[70630]: 37665f477d8460617beb522bc4914e37884ba9b73b0a7d81265eda0c1351c115 2024-05-03T20:33:01.463 INFO:journalctl@ceph.mds.j.smithi181.stdout:May 03 20:33:00 smithi181 systemd[1]: Started Ceph mds.j for 1cd058ee-098b-11ef-bc97-c7b262605968. 2024-05-03T20:33:01.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:01 smithi204 ceph-mon[33595]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:01.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:01 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:01.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:01 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:01.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:01 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:01.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:01 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:33:01.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:01 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:33:01.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:01 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:01.918 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:01 smithi181 ceph-mon[29801]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:01.918 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:01 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:01.918 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:01 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:01.918 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:01 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:01.918 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:01 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:33:01.919 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:01 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:33:01.919 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:01 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:02.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:01 smithi196 ceph-mon[32858]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:02.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:01 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:02.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:01 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:02.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:01 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:02.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:01 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:33:02.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:01 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:33:02.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:01 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:02.434 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T20:33:02.707 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:02 smithi181 ceph-mon[29801]: Deploying daemon mds.d on smithi181 2024-05-03T20:33:02.707 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:02 smithi181 ceph-mon[29801]: mds.? [v2:172.21.15.181:6834/2414172454,v1:172.21.15.181:6835/2414172454] up:boot 2024-05-03T20:33:02.708 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:02 smithi181 ceph-mon[29801]: fsmap 3 up:standby 2024-05-03T20:33:02.708 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:02 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-03T20:33:02.797 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:33:03.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:02 smithi196 ceph-mon[32858]: Deploying daemon mds.d on smithi181 2024-05-03T20:33:03.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:02 smithi196 ceph-mon[32858]: mds.? [v2:172.21.15.181:6834/2414172454,v1:172.21.15.181:6835/2414172454] up:boot 2024-05-03T20:33:03.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:02 smithi196 ceph-mon[32858]: fsmap 3 up:standby 2024-05-03T20:33:03.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:02 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-03T20:33:03.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:02 smithi204 ceph-mon[33595]: Deploying daemon mds.d on smithi181 2024-05-03T20:33:03.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:02 smithi204 ceph-mon[33595]: mds.? [v2:172.21.15.181:6834/2414172454,v1:172.21.15.181:6835/2414172454] up:boot 2024-05-03T20:33:03.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:02 smithi204 ceph-mon[33595]: fsmap 3 up:standby 2024-05-03T20:33:03.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:02 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-03T20:33:03.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:03 smithi181 ceph-mon[29801]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:04.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:03 smithi196 ceph-mon[32858]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:04.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:03 smithi204 ceph-mon[33595]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:05.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.869+0000 7fb1f630c700 1 -- 172.21.15.181:0/223109091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1f0103ef0 msgr2=0x7fb1f01103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:05.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.869+0000 7fb1f630c700 1 --2- 172.21.15.181:0/223109091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1f0103ef0 0x7fb1f01103d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800b0a0 tx=0x7fb1e800b3b0 comp rx=0 tx=0).stop 2024-05-03T20:33:05.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.869+0000 7fb1f630c700 1 -- 172.21.15.181:0/223109091 shutdown_connections 2024-05-03T20:33:05.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.869+0000 7fb1f630c700 1 --2- 172.21.15.181:0/223109091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1f0103ef0 0x7fb1f01103d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:05.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.869+0000 7fb1f630c700 1 --2- 172.21.15.181:0/223109091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f0108460 0x7fb1f0108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:05.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.869+0000 7fb1f630c700 1 --2- 172.21.15.181:0/223109091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1f0103550 0x7fb1f01039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:05.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.869+0000 7fb1f630c700 1 -- 172.21.15.181:0/223109091 >> 172.21.15.181:0/223109091 conn(0x7fb1f007a600 msgr2=0x7fb1f007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:05.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.869+0000 7fb1f630c700 1 -- 172.21.15.181:0/223109091 shutdown_connections 2024-05-03T20:33:05.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.869+0000 7fb1f630c700 1 -- 172.21.15.181:0/223109091 wait complete. 2024-05-03T20:33:05.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.869+0000 7fb1f630c700 1 Processor -- start 2024-05-03T20:33:05.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.869+0000 7fb1f630c700 1 -- start start 2024-05-03T20:33:05.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.870+0000 7fb1f630c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f0103550 0x7fb1f019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:05.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.870+0000 7fb1f630c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1f0103ef0 0x7fb1f019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:05.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.870+0000 7fb1f630c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1f0108460 0x7fb1f01998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:05.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.870+0000 7fb1f630c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01136a0 con 0x7fb1f0103ef0 2024-05-03T20:33:05.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.870+0000 7fb1f630c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f0113520 con 0x7fb1f0108460 2024-05-03T20:33:05.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.870+0000 7fb1f630c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f0113820 con 0x7fb1f0103550 2024-05-03T20:33:05.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.870+0000 7fb1ef7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1f0103ef0 0x7fb1f019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:05.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.870+0000 7fb1ef7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1f0103ef0 0x7fb1f019fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52758/0 (socket says 172.21.15.181:52758) 2024-05-03T20:33:05.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.870+0000 7fb1ef7fe700 1 -- 172.21.15.181:0/2316029646 learned_addr learned my addr 172.21.15.181:0/2316029646 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:33:05.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.870+0000 7fb1f48a9700 1 --2- 172.21.15.181:0/2316029646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1f0108460 0x7fb1f01998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:05.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.870+0000 7fb1effff700 1 --2- 172.21.15.181:0/2316029646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f0103550 0x7fb1f019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:05.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.870+0000 7fb1ef7fe700 1 -- 172.21.15.181:0/2316029646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f0103550 msgr2=0x7fb1f019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:05.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.870+0000 7fb1ef7fe700 1 --2- 172.21.15.181:0/2316029646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f0103550 0x7fb1f019f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:05.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.870+0000 7fb1ef7fe700 1 -- 172.21.15.181:0/2316029646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1f0108460 msgr2=0x7fb1f01998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:05.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.870+0000 7fb1ef7fe700 1 --2- 172.21.15.181:0/2316029646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1f0108460 0x7fb1f01998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:05.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.870+0000 7fb1ef7fe700 1 -- 172.21.15.181:0/2316029646 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f019a040 con 0x7fb1f0103ef0 2024-05-03T20:33:05.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.870+0000 7fb1effff700 1 --2- 172.21.15.181:0/2316029646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f0103550 0x7fb1f019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:33:05.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.870+0000 7fb1ef7fe700 1 --2- 172.21.15.181:0/2316029646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1f0103ef0 0x7fb1f019fc60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb1e400d950 tx=0x7fb1e400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:05.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.871+0000 7fb1ed7fa700 1 -- 172.21.15.181:0/2316029646 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4016070 con 0x7fb1f0103ef0 2024-05-03T20:33:05.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.871+0000 7fb1f630c700 1 -- 172.21.15.181:0/2316029646 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f019a330 con 0x7fb1f0103ef0 2024-05-03T20:33:05.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.871+0000 7fb1ed7fa700 1 -- 172.21.15.181:0/2316029646 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1e4009df0 con 0x7fb1f0103ef0 2024-05-03T20:33:05.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.871+0000 7fb1ed7fa700 1 -- 172.21.15.181:0/2316029646 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e400f3c0 con 0x7fb1f0103ef0 2024-05-03T20:33:05.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.871+0000 7fb1f630c700 1 -- 172.21.15.181:0/2316029646 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f01a65f0 con 0x7fb1f0103ef0 2024-05-03T20:33:05.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.872+0000 7fb1f630c700 1 -- 172.21.15.181:0/2316029646 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1f006a190 con 0x7fb1f0103ef0 2024-05-03T20:33:05.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.873+0000 7fb1ed7fa700 1 -- 172.21.15.181:0/2316029646 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fb1e4019070 con 0x7fb1f0103ef0 2024-05-03T20:33:05.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.873+0000 7fb1ed7fa700 1 --2- 172.21.15.181:0/2316029646 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb1d80b2880 0x7fb1d80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:05.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.874+0000 7fb1effff700 1 --2- 172.21.15.181:0/2316029646 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb1d80b2880 0x7fb1d80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:05.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.874+0000 7fb1ed7fa700 1 -- 172.21.15.181:0/2316029646 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fb1e40e1990 con 0x7fb1f0103ef0 2024-05-03T20:33:05.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.874+0000 7fb1effff700 1 --2- 172.21.15.181:0/2316029646 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb1d80b2880 0x7fb1d80b4d40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0005e90 tx=0x7fb1e0005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:05.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:04.877+0000 7fb1ed7fa700 1 -- 172.21.15.181:0/2316029646 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb1e40a9b50 con 0x7fb1f0103ef0 2024-05-03T20:33:05.054 INFO:journalctl@ceph.mds.d.smithi181.stdout:May 03 20:33:04 smithi181 systemd[1]: Started Ceph mds.d for 1cd058ee-098b-11ef-bc97-c7b262605968. 2024-05-03T20:33:05.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:05.147+0000 7fb1f630c700 1 -- 172.21.15.181:0/2316029646 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1f019ae10 con 0x7fb1f0103ef0 2024-05-03T20:33:05.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:05.148+0000 7fb1ed7fa700 1 -- 172.21.15.181:0/2316029646 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3052 (secure 0 0 0) 0x7fb1e40a9450 con 0x7fb1f0103ef0 2024-05-03T20:33:05.148 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:33:05.148 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2414172454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2414172454},{"type":"v1","addr":"172.21.15.181:6835","nonce":2414172454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3975688022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3975688022},{"type":"v1","addr":"172.21.15.196:6833","nonce":3975688022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2409628823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2409628823},{"type":"v1","addr":"172.21.15.204:6833","nonce":2409628823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-03T20:33:05.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:05.150+0000 7fb1f630c700 1 -- 172.21.15.181:0/2316029646 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb1d80b2880 msgr2=0x7fb1d80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:05.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:05.150+0000 7fb1f630c700 1 --2- 172.21.15.181:0/2316029646 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb1d80b2880 0x7fb1d80b4d40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0005e90 tx=0x7fb1e0005da0 comp rx=0 tx=0).stop 2024-05-03T20:33:05.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:05.150+0000 7fb1f630c700 1 -- 172.21.15.181:0/2316029646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1f0103ef0 msgr2=0x7fb1f019fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:05.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:05.150+0000 7fb1f630c700 1 --2- 172.21.15.181:0/2316029646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1f0103ef0 0x7fb1f019fc60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb1e400d950 tx=0x7fb1e400dc60 comp rx=0 tx=0).stop 2024-05-03T20:33:05.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:05.150+0000 7fb1f630c700 1 -- 172.21.15.181:0/2316029646 shutdown_connections 2024-05-03T20:33:05.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:05.150+0000 7fb1f630c700 1 --2- 172.21.15.181:0/2316029646 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb1d80b2880 0x7fb1d80b4d40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:05.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:05.150+0000 7fb1f630c700 1 --2- 172.21.15.181:0/2316029646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1f0108460 0x7fb1f01998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:05.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:05.150+0000 7fb1f630c700 1 --2- 172.21.15.181:0/2316029646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f0103550 0x7fb1f019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:05.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:05.150+0000 7fb1f630c700 1 --2- 172.21.15.181:0/2316029646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1f0103ef0 0x7fb1f019fc60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:05.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:05.150+0000 7fb1f630c700 1 -- 172.21.15.181:0/2316029646 >> 172.21.15.181:0/2316029646 conn(0x7fb1f007a600 msgr2=0x7fb1f01015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:05.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:05.150+0000 7fb1f630c700 1 -- 172.21.15.181:0/2316029646 shutdown_connections 2024-05-03T20:33:05.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:05.150+0000 7fb1f630c700 1 -- 172.21.15.181:0/2316029646 wait complete. 2024-05-03T20:33:05.151 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 4 2024-05-03T20:33:05.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:05 smithi181 ceph-mon[29801]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:05.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:05 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:05.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:05 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:05.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:05 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:05.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:05 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:33:05.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:05 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:33:05.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:05 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:05.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:05 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/2316029646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:33:05.926 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:05 smithi196 ceph-mon[32858]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:05.926 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:05 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:05.926 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:05 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:05.926 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:05 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:05.926 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:05 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:33:05.926 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:05 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:33:05.926 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:05 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:05.926 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:05 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/2316029646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:33:06.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:05 smithi204 ceph-mon[33595]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:06.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:05 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:06.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:05 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:06.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:05 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:06.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:05 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:33:06.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:05 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:33:06.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:05 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:06.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:05 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/2316029646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:33:06.806 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:06 smithi196 ceph-mon[32858]: Deploying daemon mds.e on smithi196 2024-05-03T20:33:06.806 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:06 smithi196 ceph-mon[32858]: mds.? [v2:172.21.15.181:6836/204127189,v1:172.21.15.181:6837/204127189] up:boot 2024-05-03T20:33:06.806 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:06 smithi196 ceph-mon[32858]: fsmap 4 up:standby 2024-05-03T20:33:06.807 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:06 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-03T20:33:06.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:06 smithi181 ceph-mon[29801]: Deploying daemon mds.e on smithi196 2024-05-03T20:33:06.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:06 smithi181 ceph-mon[29801]: mds.? [v2:172.21.15.181:6836/204127189,v1:172.21.15.181:6837/204127189] up:boot 2024-05-03T20:33:06.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:06 smithi181 ceph-mon[29801]: fsmap 4 up:standby 2024-05-03T20:33:06.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:06 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-03T20:33:07.107 INFO:journalctl@ceph.mds.e.smithi196.stdout:May 03 20:33:06 smithi196 systemd[1]: Starting Ceph mds.e for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:33:07.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:06 smithi204 ceph-mon[33595]: Deploying daemon mds.e on smithi196 2024-05-03T20:33:07.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:06 smithi204 ceph-mon[33595]: mds.? [v2:172.21.15.181:6836/204127189,v1:172.21.15.181:6837/204127189] up:boot 2024-05-03T20:33:07.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:06 smithi204 ceph-mon[33595]: fsmap 4 up:standby 2024-05-03T20:33:07.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:06 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-03T20:33:07.492 INFO:journalctl@ceph.mds.e.smithi196.stdout:May 03 20:33:07 smithi196 podman[62909]: 2024-05-03 20:33:07.110261794 +0000 UTC m=+0.017732055 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T20:33:07.492 INFO:journalctl@ceph.mds.e.smithi196.stdout:May 03 20:33:07 smithi196 podman[62909]: 2024-05-03 20:33:07.265682758 +0000 UTC m=+0.173152983 container create cb61e4c3e20c5a4f57c4ce075d40df901711176ef3ab2fea960b97d546b327af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-e, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T20:33:07.492 INFO:journalctl@ceph.mds.e.smithi196.stdout:May 03 20:33:07 smithi196 podman[62909]: 2024-05-03 20:33:07.493527373 +0000 UTC m=+0.400997623 container init cb61e4c3e20c5a4f57c4ce075d40df901711176ef3ab2fea960b97d546b327af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD) 2024-05-03T20:33:07.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:07 smithi196 ceph-mon[32858]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:07.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:07.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:07.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:07.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:33:07.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:33:07.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:07.857 INFO:journalctl@ceph.mds.e.smithi196.stdout:May 03 20:33:07 smithi196 podman[62909]: 2024-05-03 20:33:07.496361704 +0000 UTC m=+0.403831950 container start cb61e4c3e20c5a4f57c4ce075d40df901711176ef3ab2fea960b97d546b327af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-03T20:33:07.857 INFO:journalctl@ceph.mds.e.smithi196.stdout:May 03 20:33:07 smithi196 ceph-mds[62928]: -- 172.21.15.196:0/548152643 <== mon.2 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aef82b9a40 con 0x55aef828d000 2024-05-03T20:33:07.857 INFO:journalctl@ceph.mds.e.smithi196.stdout:May 03 20:33:07 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-e[62924]: starting mds.e at 2024-05-03T20:33:07.858 INFO:journalctl@ceph.mds.e.smithi196.stdout:May 03 20:33:07 smithi196 bash[62909]: cb61e4c3e20c5a4f57c4ce075d40df901711176ef3ab2fea960b97d546b327af 2024-05-03T20:33:07.858 INFO:journalctl@ceph.mds.e.smithi196.stdout:May 03 20:33:07 smithi196 systemd[1]: Started Ceph mds.e for 1cd058ee-098b-11ef-bc97-c7b262605968. 2024-05-03T20:33:07.886 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T20:33:07.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:07 smithi181 ceph-mon[29801]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:07.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:07.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:07.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:07.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:33:07.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:33:07.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:08.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:07 smithi204 ceph-mon[33595]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:08.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:08.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:08.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:08.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:33:08.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:33:08.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:08.140 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:33:08.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:08 smithi204 ceph-mon[33595]: Deploying daemon mds.f on smithi204 2024-05-03T20:33:08.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:08 smithi204 ceph-mon[33595]: mds.? [v2:172.21.15.196:6834/1820206106,v1:172.21.15.196:6835/1820206106] up:boot 2024-05-03T20:33:08.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:08 smithi204 ceph-mon[33595]: fsmap 5 up:standby 2024-05-03T20:33:08.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:08 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-03T20:33:08.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:08 smithi181 ceph-mon[29801]: Deploying daemon mds.f on smithi204 2024-05-03T20:33:08.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:08 smithi181 ceph-mon[29801]: mds.? [v2:172.21.15.196:6834/1820206106,v1:172.21.15.196:6835/1820206106] up:boot 2024-05-03T20:33:08.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:08 smithi181 ceph-mon[29801]: fsmap 5 up:standby 2024-05-03T20:33:08.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:08 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-03T20:33:09.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:08 smithi196 ceph-mon[32858]: Deploying daemon mds.f on smithi204 2024-05-03T20:33:09.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:08 smithi196 ceph-mon[32858]: mds.? [v2:172.21.15.196:6834/1820206106,v1:172.21.15.196:6835/1820206106] up:boot 2024-05-03T20:33:09.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:08 smithi196 ceph-mon[32858]: fsmap 5 up:standby 2024-05-03T20:33:09.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:08 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-03T20:33:09.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.649+0000 7f321b366700 1 -- 172.21.15.181:0/795600665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321406c820 msgr2=0x7f32141080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:09.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.649+0000 7f321b366700 1 --2- 172.21.15.181:0/795600665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321406c820 0x7f32141080e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f32100099c0 tx=0x7f3210009cd0 comp rx=0 tx=0).stop 2024-05-03T20:33:09.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.649+0000 7f321b366700 1 -- 172.21.15.181:0/795600665 shutdown_connections 2024-05-03T20:33:09.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.649+0000 7f321b366700 1 --2- 172.21.15.181:0/795600665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32141087c0 0x7f32141103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:09.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.649+0000 7f321b366700 1 --2- 172.21.15.181:0/795600665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f321406be70 0x7f321406c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:09.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.649+0000 7f321b366700 1 --2- 172.21.15.181:0/795600665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321406c820 0x7f32141080e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:09.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.649+0000 7f321b366700 1 -- 172.21.15.181:0/795600665 >> 172.21.15.181:0/795600665 conn(0x7f32140783f0 msgr2=0x7f3214078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:09.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.649+0000 7f321b366700 1 -- 172.21.15.181:0/795600665 shutdown_connections 2024-05-03T20:33:09.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.649+0000 7f321b366700 1 -- 172.21.15.181:0/795600665 wait complete. 2024-05-03T20:33:09.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.649+0000 7f321b366700 1 Processor -- start 2024-05-03T20:33:09.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.650+0000 7f321b366700 1 -- start start 2024-05-03T20:33:09.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.650+0000 7f321b366700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f321406be70 0x7f321419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:09.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.650+0000 7f321b366700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321406c820 0x7f321419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:09.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.650+0000 7f321b366700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32141087c0 0x7f3214199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:09.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.650+0000 7f321b366700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3214113670 con 0x7f321406c820 2024-05-03T20:33:09.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.650+0000 7f321b366700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f32141134f0 con 0x7f32141087c0 2024-05-03T20:33:09.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.650+0000 7f321b366700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f32141137f0 con 0x7f321406be70 2024-05-03T20:33:09.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.650+0000 7f3218901700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321406c820 0x7f321419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:09.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.650+0000 7f3218901700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321406c820 0x7f321419fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34384/0 (socket says 172.21.15.181:34384) 2024-05-03T20:33:09.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.650+0000 7f3218901700 1 -- 172.21.15.181:0/2132533960 learned_addr learned my addr 172.21.15.181:0/2132533960 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:33:09.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.650+0000 7f3219102700 1 --2- 172.21.15.181:0/2132533960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f321406be70 0x7f321419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:09.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.650+0000 7f3219903700 1 --2- 172.21.15.181:0/2132533960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32141087c0 0x7f3214199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:09.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.650+0000 7f3218901700 1 -- 172.21.15.181:0/2132533960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f321406be70 msgr2=0x7f321419f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:09.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.650+0000 7f3218901700 1 --2- 172.21.15.181:0/2132533960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f321406be70 0x7f321419f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:09.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.650+0000 7f3218901700 1 -- 172.21.15.181:0/2132533960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32141087c0 msgr2=0x7f3214199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:09.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.650+0000 7f3218901700 1 --2- 172.21.15.181:0/2132533960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32141087c0 0x7f3214199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:09.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.650+0000 7f3218901700 1 -- 172.21.15.181:0/2132533960 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321419a0f0 con 0x7f321406c820 2024-05-03T20:33:09.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.650+0000 7f3218901700 1 --2- 172.21.15.181:0/2132533960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321406c820 0x7f321419fd10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3210006010 tx=0x7f32100047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:09.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.651+0000 7f320a7fc700 1 -- 172.21.15.181:0/2132533960 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3210027070 con 0x7f321406c820 2024-05-03T20:33:09.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.651+0000 7f320a7fc700 1 -- 172.21.15.181:0/2132533960 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3210003f20 con 0x7f321406c820 2024-05-03T20:33:09.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.651+0000 7f321b366700 1 -- 172.21.15.181:0/2132533960 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f321419a380 con 0x7f321406c820 2024-05-03T20:33:09.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.651+0000 7f320a7fc700 1 -- 172.21.15.181:0/2132533960 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3210022c60 con 0x7f321406c820 2024-05-03T20:33:09.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.651+0000 7f321b366700 1 -- 172.21.15.181:0/2132533960 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32141a66a0 con 0x7f321406c820 2024-05-03T20:33:09.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.652+0000 7f321b366700 1 -- 172.21.15.181:0/2132533960 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f321406a190 con 0x7f321406c820 2024-05-03T20:33:09.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.654+0000 7f320a7fc700 1 -- 172.21.15.181:0/2132533960 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f321002a070 con 0x7f321406c820 2024-05-03T20:33:09.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.654+0000 7f320a7fc700 1 --2- 172.21.15.181:0/2132533960 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f32000b2900 0x7f32000b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:09.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.654+0000 7f3219102700 1 --2- 172.21.15.181:0/2132533960 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f32000b2900 0x7f32000b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:09.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.654+0000 7f320a7fc700 1 -- 172.21.15.181:0/2132533960 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f3210022e00 con 0x7f321406c820 2024-05-03T20:33:09.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.655+0000 7f3219102700 1 --2- 172.21.15.181:0/2132533960 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f32000b2900 0x7f32000b4dc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f321419aad0 tx=0x7f320400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:09.657 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.658+0000 7f320a7fc700 1 -- 172.21.15.181:0/2132533960 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3210031070 con 0x7f321406c820 2024-05-03T20:33:09.848 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:09 smithi204 ceph-mon[33595]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:09.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:09 smithi196 ceph-mon[32858]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:09.907 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.907+0000 7f321b366700 1 -- 172.21.15.181:0/2132533960 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f321402e510 con 0x7f321406c820 2024-05-03T20:33:09.907 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.907+0000 7f320a7fc700 1 -- 172.21.15.181:0/2132533960 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4705 (secure 0 0 0) 0x7f3210013070 con 0x7f321406c820 2024-05-03T20:33:09.907 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:33:09.907 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2414172454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2414172454},{"type":"v1","addr":"172.21.15.181:6835","nonce":2414172454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/204127189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":204127189},{"type":"v1","addr":"172.21.15.181:6837","nonce":204127189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3975688022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3975688022},{"type":"v1","addr":"172.21.15.196:6833","nonce":3975688022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2409628823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2409628823},{"type":"v1","addr":"172.21.15.204:6833","nonce":2409628823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1820206106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1820206106},{"type":"v1","addr":"172.21.15.196:6835","nonce":1820206106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-03T20:33:09.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.910+0000 7f321b366700 1 -- 172.21.15.181:0/2132533960 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f32000b2900 msgr2=0x7f32000b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:09.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.910+0000 7f321b366700 1 --2- 172.21.15.181:0/2132533960 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f32000b2900 0x7f32000b4dc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f321419aad0 tx=0x7f320400b410 comp rx=0 tx=0).stop 2024-05-03T20:33:09.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.910+0000 7f321b366700 1 -- 172.21.15.181:0/2132533960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321406c820 msgr2=0x7f321419fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:09.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.910+0000 7f321b366700 1 --2- 172.21.15.181:0/2132533960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321406c820 0x7f321419fd10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3210006010 tx=0x7f32100047b0 comp rx=0 tx=0).stop 2024-05-03T20:33:09.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.910+0000 7f321b366700 1 -- 172.21.15.181:0/2132533960 shutdown_connections 2024-05-03T20:33:09.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.910+0000 7f321b366700 1 --2- 172.21.15.181:0/2132533960 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f32000b2900 0x7f32000b4dc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:09.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.910+0000 7f321b366700 1 --2- 172.21.15.181:0/2132533960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32141087c0 0x7f3214199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:09.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.910+0000 7f321b366700 1 --2- 172.21.15.181:0/2132533960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f321406be70 0x7f321419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:09.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.910+0000 7f321b366700 1 --2- 172.21.15.181:0/2132533960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321406c820 0x7f321419fd10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:09.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.910+0000 7f321b366700 1 -- 172.21.15.181:0/2132533960 >> 172.21.15.181:0/2132533960 conn(0x7f32140783f0 msgr2=0x7f3214107400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:09.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.910+0000 7f321b366700 1 -- 172.21.15.181:0/2132533960 shutdown_connections 2024-05-03T20:33:09.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:09.910+0000 7f321b366700 1 -- 172.21.15.181:0/2132533960 wait complete. 2024-05-03T20:33:09.910 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 6 2024-05-03T20:33:09.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:09 smithi181 ceph-mon[29801]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:10.109 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 03 20:33:10 smithi204 podman[63928]: 2024-05-03 20:33:10.087941369 +0000 UTC m=+0.331891853 container start a4aca1db7f9da0393c9ed052931f4d118e629e43dcb57d7b48eb6c57a8a3e947 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430) 2024-05-03T20:33:10.363 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 03 20:33:10 smithi204 ceph-mds[63947]: -- 172.21.15.204:0/2359645918 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cc646e5a40 con 0x55cc646b8800 2024-05-03T20:33:10.363 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 03 20:33:10 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-f[63943]: starting mds.f at 2024-05-03T20:33:10.363 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 03 20:33:10 smithi204 bash[63928]: a4aca1db7f9da0393c9ed052931f4d118e629e43dcb57d7b48eb6c57a8a3e947 2024-05-03T20:33:10.363 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 03 20:33:10 smithi204 systemd[1]: Started Ceph mds.f for 1cd058ee-098b-11ef-bc97-c7b262605968. 2024-05-03T20:33:10.866 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:10 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/2132533960' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:33:10.866 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:10 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:10.866 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:10 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:10.866 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:10 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:10.866 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:10 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:33:10.866 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:10 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:33:10.866 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:10 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:11.005 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:10 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/2132533960' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:33:11.005 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:10 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:11.005 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:10 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:11.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:10 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:11.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:10 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:33:11.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:10 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:33:11.006 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:10 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:11.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:10 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/2132533960' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:33:11.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:10 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:11.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:10 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:11.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:10 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:11.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:10 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:33:11.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:10 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:33:11.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:10 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:11.924 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:11 smithi204 ceph-mon[33595]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:11.924 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:11 smithi204 ceph-mon[33595]: Deploying daemon mds.c on smithi204 2024-05-03T20:33:11.925 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:11 smithi204 ceph-mon[33595]: mds.? [v2:172.21.15.204:6834/3870850287,v1:172.21.15.204:6835/3870850287] up:boot 2024-05-03T20:33:11.925 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:11 smithi204 ceph-mon[33595]: fsmap 6 up:standby 2024-05-03T20:33:11.925 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:11 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T20:33:11.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:11 smithi181 ceph-mon[29801]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:11.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:11 smithi181 ceph-mon[29801]: Deploying daemon mds.c on smithi204 2024-05-03T20:33:11.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:11 smithi181 ceph-mon[29801]: mds.? [v2:172.21.15.204:6834/3870850287,v1:172.21.15.204:6835/3870850287] up:boot 2024-05-03T20:33:11.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:11 smithi181 ceph-mon[29801]: fsmap 6 up:standby 2024-05-03T20:33:11.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:11 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T20:33:12.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:11 smithi196 ceph-mon[32858]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:12.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:11 smithi196 ceph-mon[32858]: Deploying daemon mds.c on smithi204 2024-05-03T20:33:12.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:11 smithi196 ceph-mon[32858]: mds.? [v2:172.21.15.204:6834/3870850287,v1:172.21.15.204:6835/3870850287] up:boot 2024-05-03T20:33:12.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:11 smithi196 ceph-mon[32858]: fsmap 6 up:standby 2024-05-03T20:33:12.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:11 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T20:33:12.598 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T20:33:12.604 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 03 20:33:12 smithi204 systemd[1]: Starting Ceph mds.c for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:33:12.852 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:33:12.863 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 03 20:33:12 smithi204 podman[64233]: 2024-05-03 20:33:12.508543487 +0000 UTC m=+0.018431019 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T20:33:12.863 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 03 20:33:12 smithi204 podman[64233]: 2024-05-03 20:33:12.611768309 +0000 UTC m=+0.121655867 container create df2ea6b1690e1992ee18250f1f41446524f127dee49b85d7e21726e4ae7ffc56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-c, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-03T20:33:12.864 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 03 20:33:12 smithi204 podman[64233]: 2024-05-03 20:33:12.849286904 +0000 UTC m=+0.359174420 container init df2ea6b1690e1992ee18250f1f41446524f127dee49b85d7e21726e4ae7ffc56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T20:33:12.864 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 03 20:33:12 smithi204 podman[64233]: 2024-05-03 20:33:12.852045776 +0000 UTC m=+0.361933305 container start df2ea6b1690e1992ee18250f1f41446524f127dee49b85d7e21726e4ae7ffc56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-03T20:33:13.363 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 03 20:33:12 smithi204 ceph-mds[64252]: -- 172.21.15.204:0/2837439823 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562612f27a40 con 0x562612efa800 2024-05-03T20:33:13.363 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 03 20:33:12 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-c[64248]: starting mds.c at 2024-05-03T20:33:13.363 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 03 20:33:12 smithi204 bash[64233]: df2ea6b1690e1992ee18250f1f41446524f127dee49b85d7e21726e4ae7ffc56 2024-05-03T20:33:13.363 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 03 20:33:12 smithi204 systemd[1]: Started Ceph mds.c for 1cd058ee-098b-11ef-bc97-c7b262605968. 2024-05-03T20:33:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:13 smithi196 ceph-mon[32858]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:13 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:13 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:13 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:13 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:33:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:13 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:33:13.944 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:13 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:14.015 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:13 smithi181 ceph-mon[29801]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:14.015 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:13 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:14.016 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:13 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:14.016 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:13 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:14.016 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:13 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:33:14.016 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:13 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:33:14.016 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:13 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:14.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:13 smithi204 ceph-mon[33595]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:14.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:13 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:14.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:13 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:14.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:13 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:14.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:13 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:33:14.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:13 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:33:14.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:13 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:14.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.408+0000 7f158ea92700 1 -- 172.21.15.181:0/1141392609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1588108560 msgr2=0x7f1588108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:14.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.408+0000 7f158ea92700 1 --2- 172.21.15.181:0/1141392609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1588108560 0x7f1588108940 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f157c009990 tx=0x7f157c009ca0 comp rx=0 tx=0).stop 2024-05-03T20:33:14.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.408+0000 7f158ea92700 1 -- 172.21.15.181:0/1141392609 shutdown_connections 2024-05-03T20:33:14.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.408+0000 7f158ea92700 1 --2- 172.21.15.181:0/1141392609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1588102800 0x7f1588110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:14.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.408+0000 7f158ea92700 1 --2- 172.21.15.181:0/1141392609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1588108560 0x7f1588108940 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:14.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.408+0000 7f158ea92700 1 --2- 172.21.15.181:0/1141392609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1588101e40 0x7f15881022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:14.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.408+0000 7f158ea92700 1 -- 172.21.15.181:0/1141392609 >> 172.21.15.181:0/1141392609 conn(0x7f15880776c0 msgr2=0x7f1588077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:14.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.408+0000 7f158ea92700 1 -- 172.21.15.181:0/1141392609 shutdown_connections 2024-05-03T20:33:14.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.408+0000 7f158ea92700 1 -- 172.21.15.181:0/1141392609 wait complete. 2024-05-03T20:33:14.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.409+0000 7f158ea92700 1 Processor -- start 2024-05-03T20:33:14.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.409+0000 7f158ea92700 1 -- start start 2024-05-03T20:33:14.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.409+0000 7f158ea92700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1588101e40 0x7f158819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:14.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.409+0000 7f158ea92700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1588102800 0x7f158819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:14.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.409+0000 7f158ea92700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1588108560 0x7f15881999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:14.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.409+0000 7f158ea92700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1588113670 con 0x7f1588108560 2024-05-03T20:33:14.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.409+0000 7f158ea92700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f15881134f0 con 0x7f1588101e40 2024-05-03T20:33:14.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.409+0000 7f158ea92700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f15881137f0 con 0x7f1588102800 2024-05-03T20:33:14.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.409+0000 7f158c82e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1588101e40 0x7f158819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:14.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.409+0000 7f158d02f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1588108560 0x7f15881999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:14.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.409+0000 7f158c82e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1588101e40 0x7f158819f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.181:51398/0 (socket says 172.21.15.181:51398) 2024-05-03T20:33:14.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.409+0000 7f158c82e700 1 -- 172.21.15.181:0/3212635697 learned_addr learned my addr 172.21.15.181:0/3212635697 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:33:14.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.409+0000 7f1587fff700 1 --2- 172.21.15.181:0/3212635697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1588102800 0x7f158819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:14.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.409+0000 7f158c82e700 1 -- 172.21.15.181:0/3212635697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1588102800 msgr2=0x7f158819fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:14.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.409+0000 7f158c82e700 1 --2- 172.21.15.181:0/3212635697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1588102800 0x7f158819fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:14.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.409+0000 7f158c82e700 1 -- 172.21.15.181:0/3212635697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1588108560 msgr2=0x7f15881999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:14.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.409+0000 7f158c82e700 1 --2- 172.21.15.181:0/3212635697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1588108560 0x7f15881999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:14.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.410+0000 7f158c82e700 1 -- 172.21.15.181:0/3212635697 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158819a110 con 0x7f1588101e40 2024-05-03T20:33:14.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.410+0000 7f158d02f700 1 --2- 172.21.15.181:0/3212635697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1588108560 0x7f15881999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T20:33:14.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.410+0000 7f158c82e700 1 --2- 172.21.15.181:0/3212635697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1588101e40 0x7f158819f7f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f157c000c00 tx=0x7f157c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:14.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.410+0000 7f1585ffb700 1 -- 172.21.15.181:0/3212635697 <== mon.2 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c027070 con 0x7f1588101e40 2024-05-03T20:33:14.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.410+0000 7f158ea92700 1 -- 172.21.15.181:0/3212635697 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f158819a3a0 con 0x7f1588101e40 2024-05-03T20:33:14.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.410+0000 7f158ea92700 1 -- 172.21.15.181:0/3212635697 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15881a66c0 con 0x7f1588101e40 2024-05-03T20:33:14.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.410+0000 7f1585ffb700 1 -- 172.21.15.181:0/3212635697 <== mon.2 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f157c022470 con 0x7f1588101e40 2024-05-03T20:33:14.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.410+0000 7f1585ffb700 1 -- 172.21.15.181:0/3212635697 <== mon.2 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c017650 con 0x7f1588101e40 2024-05-03T20:33:14.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.411+0000 7f158ea92700 1 -- 172.21.15.181:0/3212635697 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f158806a190 con 0x7f1588101e40 2024-05-03T20:33:14.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.412+0000 7f1585ffb700 1 -- 172.21.15.181:0/3212635697 <== mon.2 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f157c02a070 con 0x7f1588101e40 2024-05-03T20:33:14.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.413+0000 7f1585ffb700 1 --2- 172.21.15.181:0/3212635697 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f15780b2880 0x7f15780b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:14.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.413+0000 7f1587fff700 1 --2- 172.21.15.181:0/3212635697 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f15780b2880 0x7f15780b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:14.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.413+0000 7f1585ffb700 1 -- 172.21.15.181:0/3212635697 <== mon.2 v2:172.21.15.196:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f157c0e2b30 con 0x7f1588101e40 2024-05-03T20:33:14.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.413+0000 7f1587fff700 1 --2- 172.21.15.181:0/3212635697 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f15780b2880 0x7f15780b4d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1588077cc0 tx=0x7f157400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:14.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.418+0000 7f1585ffb700 1 -- 172.21.15.181:0/3212635697 <== mon.2 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f157c013070 con 0x7f1588101e40 2024-05-03T20:33:14.673 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.675+0000 7f158ea92700 1 -- 172.21.15.181:0/3212635697 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f158819aee0 con 0x7f1588101e40 2024-05-03T20:33:14.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.675+0000 7f1585ffb700 1 -- 172.21.15.181:0/3212635697 <== mon.2 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6361 (secure 0 0 0) 0x7f157c0aace0 con 0x7f1588101e40 2024-05-03T20:33:14.674 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:33:14.674 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2414172454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2414172454},{"type":"v1","addr":"172.21.15.181:6835","nonce":2414172454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/204127189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":204127189},{"type":"v1","addr":"172.21.15.181:6837","nonce":204127189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3975688022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3975688022},{"type":"v1","addr":"172.21.15.196:6833","nonce":3975688022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2409628823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2409628823},{"type":"v1","addr":"172.21.15.204:6833","nonce":2409628823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1820206106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1820206106},{"type":"v1","addr":"172.21.15.196:6835","nonce":1820206106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3870850287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3870850287},{"type":"v1","addr":"172.21.15.204:6835","nonce":3870850287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3006728559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3006728559},{"type":"v1","addr":"172.21.15.204:6837","nonce":3006728559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-03T20:33:14.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.678+0000 7f158ea92700 1 -- 172.21.15.181:0/3212635697 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f15780b2880 msgr2=0x7f15780b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:14.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.678+0000 7f158ea92700 1 --2- 172.21.15.181:0/3212635697 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f15780b2880 0x7f15780b4d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1588077cc0 tx=0x7f157400b3c0 comp rx=0 tx=0).stop 2024-05-03T20:33:14.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.678+0000 7f158ea92700 1 -- 172.21.15.181:0/3212635697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1588101e40 msgr2=0x7f158819f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:14.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.678+0000 7f158ea92700 1 --2- 172.21.15.181:0/3212635697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1588101e40 0x7f158819f7f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f157c000c00 tx=0x7f157c009580 comp rx=0 tx=0).stop 2024-05-03T20:33:14.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.678+0000 7f158ea92700 1 -- 172.21.15.181:0/3212635697 shutdown_connections 2024-05-03T20:33:14.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.678+0000 7f158ea92700 1 --2- 172.21.15.181:0/3212635697 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f15780b2880 0x7f15780b4d40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:14.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.678+0000 7f158ea92700 1 --2- 172.21.15.181:0/3212635697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1588108560 0x7f15881999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:14.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.678+0000 7f158ea92700 1 --2- 172.21.15.181:0/3212635697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1588101e40 0x7f158819f7f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:14.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.678+0000 7f158ea92700 1 --2- 172.21.15.181:0/3212635697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1588102800 0x7f158819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:14.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.678+0000 7f158ea92700 1 -- 172.21.15.181:0/3212635697 >> 172.21.15.181:0/3212635697 conn(0x7f15880776c0 msgr2=0x7f1588100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:14.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.678+0000 7f158ea92700 1 -- 172.21.15.181:0/3212635697 shutdown_connections 2024-05-03T20:33:14.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:14.678+0000 7f158ea92700 1 -- 172.21.15.181:0/3212635697 wait complete. 2024-05-03T20:33:14.678 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 8 2024-05-03T20:33:14.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:14 smithi181 ceph-mon[29801]: Deploying daemon mds.h on smithi196 2024-05-03T20:33:14.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:14 smithi181 ceph-mon[29801]: mds.? [v2:172.21.15.204:6836/3006728559,v1:172.21.15.204:6837/3006728559] up:boot 2024-05-03T20:33:14.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:14 smithi181 ceph-mon[29801]: fsmap 7 up:standby 2024-05-03T20:33:14.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:14 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T20:33:14.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:14 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3212635697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:33:15.014 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:14 smithi196 ceph-mon[32858]: Deploying daemon mds.h on smithi196 2024-05-03T20:33:15.014 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:14 smithi196 ceph-mon[32858]: mds.? [v2:172.21.15.204:6836/3006728559,v1:172.21.15.204:6837/3006728559] up:boot 2024-05-03T20:33:15.014 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:14 smithi196 ceph-mon[32858]: fsmap 7 up:standby 2024-05-03T20:33:15.014 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:14 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T20:33:15.014 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:14 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3212635697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:33:15.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:14 smithi204 ceph-mon[33595]: Deploying daemon mds.h on smithi196 2024-05-03T20:33:15.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:14 smithi204 ceph-mon[33595]: mds.? [v2:172.21.15.204:6836/3006728559,v1:172.21.15.204:6837/3006728559] up:boot 2024-05-03T20:33:15.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:14 smithi204 ceph-mon[33595]: fsmap 7 up:standby 2024-05-03T20:33:15.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:14 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T20:33:15.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:14 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3212635697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:33:15.357 INFO:journalctl@ceph.mds.h.smithi196.stdout:May 03 20:33:15 smithi196 systemd[1]: Starting Ceph mds.h for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:33:15.710 INFO:journalctl@ceph.mds.h.smithi196.stdout:May 03 20:33:15 smithi196 podman[63220]: 2024-05-03 20:33:15.315038332 +0000 UTC m=+0.018170948 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T20:33:15.711 INFO:journalctl@ceph.mds.h.smithi196.stdout:May 03 20:33:15 smithi196 podman[63220]: 2024-05-03 20:33:15.41840585 +0000 UTC m=+0.121538440 container create 2f6a0b1b3e94588ec9c390caa51b67475fce66d4330833638def031dee65b837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T20:33:15.711 INFO:journalctl@ceph.mds.h.smithi196.stdout:May 03 20:33:15 smithi196 podman[63220]: 2024-05-03 20:33:15.629102739 +0000 UTC m=+0.332235330 container init 2f6a0b1b3e94588ec9c390caa51b67475fce66d4330833638def031dee65b837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240430, io.buildah.version=1.33.5) 2024-05-03T20:33:15.711 INFO:journalctl@ceph.mds.h.smithi196.stdout:May 03 20:33:15 smithi196 podman[63220]: 2024-05-03 20:33:15.631967818 +0000 UTC m=+0.335100417 container start 2f6a0b1b3e94588ec9c390caa51b67475fce66d4330833638def031dee65b837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, ceph=True) 2024-05-03T20:33:15.711 INFO:journalctl@ceph.mds.h.smithi196.stdout:May 03 20:33:15 smithi196 ceph-mds[63240]: -- 172.21.15.196:0/2484670740 <== mon.2 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5647c06cda40 con 0x5647c06a0c00 2024-05-03T20:33:15.711 INFO:journalctl@ceph.mds.h.smithi196.stdout:May 03 20:33:15 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-h[63236]: starting mds.h at 2024-05-03T20:33:15.711 INFO:journalctl@ceph.mds.h.smithi196.stdout:May 03 20:33:15 smithi196 bash[63220]: 2f6a0b1b3e94588ec9c390caa51b67475fce66d4330833638def031dee65b837 2024-05-03T20:33:15.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:15 smithi204 ceph-mon[33595]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:16.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:15 smithi196 ceph-mon[32858]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:16.107 INFO:journalctl@ceph.mds.h.smithi196.stdout:May 03 20:33:15 smithi196 systemd[1]: Started Ceph mds.h for 1cd058ee-098b-11ef-bc97-c7b262605968. 2024-05-03T20:33:16.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:15 smithi181 ceph-mon[29801]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:17.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:16 smithi196 ceph-mon[32858]: mds.? [v2:172.21.15.196:6836/1529420259,v1:172.21.15.196:6837/1529420259] up:boot 2024-05-03T20:33:17.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:16 smithi196 ceph-mon[32858]: fsmap 8 up:standby 2024-05-03T20:33:17.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:16 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T20:33:17.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:16 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:17.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:16 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:17.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:16 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:17.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:16 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:33:17.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:16 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:33:17.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:16 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:17.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:16 smithi196 ceph-mon[32858]: Deploying daemon mds.i on smithi204 2024-05-03T20:33:17.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:16 smithi204 ceph-mon[33595]: mds.? [v2:172.21.15.196:6836/1529420259,v1:172.21.15.196:6837/1529420259] up:boot 2024-05-03T20:33:17.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:16 smithi204 ceph-mon[33595]: fsmap 8 up:standby 2024-05-03T20:33:17.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:16 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T20:33:17.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:16 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:17.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:16 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:17.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:16 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:17.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:16 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:33:17.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:16 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:33:17.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:16 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:17.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:16 smithi204 ceph-mon[33595]: Deploying daemon mds.i on smithi204 2024-05-03T20:33:17.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:16 smithi181 ceph-mon[29801]: mds.? [v2:172.21.15.196:6836/1529420259,v1:172.21.15.196:6837/1529420259] up:boot 2024-05-03T20:33:17.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:16 smithi181 ceph-mon[29801]: fsmap 8 up:standby 2024-05-03T20:33:17.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:16 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T20:33:17.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:16 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:17.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:16 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:17.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:16 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:17.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:16 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:33:17.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:16 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:33:17.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:16 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:17.214 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:16 smithi181 ceph-mon[29801]: Deploying daemon mds.i on smithi204 2024-05-03T20:33:17.315 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T20:33:17.566 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:33:17.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:17 smithi181 ceph-mon[29801]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:18.064 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:17 smithi204 ceph-mon[33595]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:18.064 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 03 20:33:17 smithi204 systemd[1]: Starting Ceph mds.i for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:33:18.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:17 smithi196 ceph-mon[32858]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:18.334 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 03 20:33:18 smithi204 podman[64555]: 2024-05-03 20:33:18.065560017 +0000 UTC m=+0.102691260 container create 23106480b45ab8d258d63bec65bfa2e99c7e3cfbfbd88cff081f693a96d6bac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-03T20:33:18.334 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 03 20:33:18 smithi204 podman[64555]: 2024-05-03 20:33:17.981107074 +0000 UTC m=+0.018238325 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T20:33:18.334 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 03 20:33:18 smithi204 podman[64555]: 2024-05-03 20:33:18.310001613 +0000 UTC m=+0.347132853 container init 23106480b45ab8d258d63bec65bfa2e99c7e3cfbfbd88cff081f693a96d6bac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-i, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD) 2024-05-03T20:33:18.334 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 03 20:33:18 smithi204 podman[64555]: 2024-05-03 20:33:18.312595007 +0000 UTC m=+0.349726247 container start 23106480b45ab8d258d63bec65bfa2e99c7e3cfbfbd88cff081f693a96d6bac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T20:33:18.613 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 03 20:33:18 smithi204 ceph-mds[64574]: -- 172.21.15.204:0/1309394099 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d731c5fa40 con 0x55d731c32c00 2024-05-03T20:33:18.613 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 03 20:33:18 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-i[64570]: starting mds.i at 2024-05-03T20:33:18.613 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 03 20:33:18 smithi204 bash[64555]: 23106480b45ab8d258d63bec65bfa2e99c7e3cfbfbd88cff081f693a96d6bac0 2024-05-03T20:33:18.613 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 03 20:33:18 smithi204 systemd[1]: Started Ceph mds.i for 1cd058ee-098b-11ef-bc97-c7b262605968. 2024-05-03T20:33:19.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.125+0000 7f7902930700 1 -- 172.21.15.181:0/1364512065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78fc1023a0 msgr2=0x7f78fc102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:19.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.125+0000 7f7902930700 1 --2- 172.21.15.181:0/1364512065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78fc1023a0 0x7f78fc102820 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f78f0009990 tx=0x7f78f0009ca0 comp rx=0 tx=0).stop 2024-05-03T20:33:19.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.125+0000 7f7902930700 1 -- 172.21.15.181:0/1364512065 shutdown_connections 2024-05-03T20:33:19.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.125+0000 7f7902930700 1 --2- 172.21.15.181:0/1364512065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78fc102d60 0x7f78fc10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:19.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.125+0000 7f7902930700 1 --2- 172.21.15.181:0/1364512065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78fc108410 0x7f78fc1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:19.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.125+0000 7f7902930700 1 --2- 172.21.15.181:0/1364512065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78fc1023a0 0x7f78fc102820 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:19.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.125+0000 7f7902930700 1 -- 172.21.15.181:0/1364512065 >> 172.21.15.181:0/1364512065 conn(0x7f78fc079350 msgr2=0x7f78fc079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:19.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.126+0000 7f7902930700 1 -- 172.21.15.181:0/1364512065 shutdown_connections 2024-05-03T20:33:19.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.126+0000 7f7902930700 1 -- 172.21.15.181:0/1364512065 wait complete. 2024-05-03T20:33:19.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.126+0000 7f7902930700 1 Processor -- start 2024-05-03T20:33:19.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.126+0000 7f7902930700 1 -- start start 2024-05-03T20:33:19.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.127+0000 7f7902930700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78fc1023a0 0x7f78fc19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:19.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.127+0000 7f7902930700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78fc102d60 0x7f78fc19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:19.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.127+0000 7f7902930700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78fc108410 0x7f78fc199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:19.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.127+0000 7f7902930700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f78fc112560 con 0x7f78fc102d60 2024-05-03T20:33:19.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.127+0000 7f7902930700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f78fc1123e0 con 0x7f78fc1023a0 2024-05-03T20:33:19.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.127+0000 7f7902930700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f78fc1126e0 con 0x7f78fc108410 2024-05-03T20:33:19.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.127+0000 7f78fb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78fc102d60 0x7f78fc19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:19.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.127+0000 7f78fb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78fc102d60 0x7f78fc19fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39676/0 (socket says 172.21.15.181:39676) 2024-05-03T20:33:19.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.127+0000 7f78fb7fe700 1 -- 172.21.15.181:0/2919826569 learned_addr learned my addr 172.21.15.181:0/2919826569 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:33:19.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.127+0000 7f78fbfff700 1 --2- 172.21.15.181:0/2919826569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78fc1023a0 0x7f78fc19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:19.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.127+0000 7f7900ecd700 1 --2- 172.21.15.181:0/2919826569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78fc108410 0x7f78fc199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:19.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.127+0000 7f78fb7fe700 1 -- 172.21.15.181:0/2919826569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78fc108410 msgr2=0x7f78fc199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:19.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.127+0000 7f78fb7fe700 1 --2- 172.21.15.181:0/2919826569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78fc108410 0x7f78fc199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:19.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.127+0000 7f78fb7fe700 1 -- 172.21.15.181:0/2919826569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78fc1023a0 msgr2=0x7f78fc19f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:19.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.127+0000 7f78fb7fe700 1 --2- 172.21.15.181:0/2919826569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78fc1023a0 0x7f78fc19f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:19.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.127+0000 7f78fb7fe700 1 -- 172.21.15.181:0/2919826569 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78fc19a0b0 con 0x7f78fc102d60 2024-05-03T20:33:19.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.127+0000 7f78fb7fe700 1 --2- 172.21.15.181:0/2919826569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78fc102d60 0x7f78fc19fcd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f78f0000c00 tx=0x7f78f00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:19.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.127+0000 7f78f97fa700 1 -- 172.21.15.181:0/2919826569 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0027070 con 0x7f78fc102d60 2024-05-03T20:33:19.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.128+0000 7f7902930700 1 -- 172.21.15.181:0/2919826569 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78fc19a340 con 0x7f78fc102d60 2024-05-03T20:33:19.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.128+0000 7f78f97fa700 1 -- 172.21.15.181:0/2919826569 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78f0003f20 con 0x7f78fc102d60 2024-05-03T20:33:19.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.128+0000 7f78f97fa700 1 -- 172.21.15.181:0/2919826569 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0022ca0 con 0x7f78fc102d60 2024-05-03T20:33:19.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.128+0000 7f7902930700 1 -- 172.21.15.181:0/2919826569 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78fc1a6660 con 0x7f78fc102d60 2024-05-03T20:33:19.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.128+0000 7f7902930700 1 -- 172.21.15.181:0/2919826569 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78fc06a190 con 0x7f78fc102d60 2024-05-03T20:33:19.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.130+0000 7f78f97fa700 1 -- 172.21.15.181:0/2919826569 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f78f002a070 con 0x7f78fc102d60 2024-05-03T20:33:19.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.131+0000 7f78f97fa700 1 --2- 172.21.15.181:0/2919826569 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f78e40b2880 0x7f78e40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:19.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.131+0000 7f78fbfff700 1 --2- 172.21.15.181:0/2919826569 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f78e40b2880 0x7f78e40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:19.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.131+0000 7f78f97fa700 1 -- 172.21.15.181:0/2919826569 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f78f00e3350 con 0x7f78fc102d60 2024-05-03T20:33:19.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.131+0000 7f78fbfff700 1 --2- 172.21.15.181:0/2919826569 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f78e40b2880 0x7f78e40b4d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f78fc19aa90 tx=0x7f78ec0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:19.133 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.135+0000 7f78f97fa700 1 -- 172.21.15.181:0/2919826569 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f78f0031070 con 0x7f78fc102d60 2024-05-03T20:33:19.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.404+0000 7f7902930700 1 -- 172.21.15.181:0/2919826569 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78fc1a6830 con 0x7f78fc102d60 2024-05-03T20:33:19.404 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.405+0000 7f78f97fa700 1 -- 172.21.15.181:0/2919826569 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8019 (secure 0 0 0) 0x7f78f0013070 con 0x7f78fc102d60 2024-05-03T20:33:19.404 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:33:19.404 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2414172454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2414172454},{"type":"v1","addr":"172.21.15.181:6835","nonce":2414172454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/204127189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":204127189},{"type":"v1","addr":"172.21.15.181:6837","nonce":204127189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3975688022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3975688022},{"type":"v1","addr":"172.21.15.196:6833","nonce":3975688022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2409628823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2409628823},{"type":"v1","addr":"172.21.15.204:6833","nonce":2409628823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1820206106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1820206106},{"type":"v1","addr":"172.21.15.196:6835","nonce":1820206106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3870850287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3870850287},{"type":"v1","addr":"172.21.15.204:6835","nonce":3870850287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3006728559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3006728559},{"type":"v1","addr":"172.21.15.204:6837","nonce":3006728559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/1529420259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1529420259},{"type":"v1","addr":"172.21.15.196:6837","nonce":1529420259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/1644828559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1644828559},{"type":"v1","addr":"172.21.15.204:6839","nonce":1644828559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-03T20:33:19.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.408+0000 7f7902930700 1 -- 172.21.15.181:0/2919826569 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f78e40b2880 msgr2=0x7f78e40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:19.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.408+0000 7f7902930700 1 --2- 172.21.15.181:0/2919826569 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f78e40b2880 0x7f78e40b4d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f78fc19aa90 tx=0x7f78ec0073d0 comp rx=0 tx=0).stop 2024-05-03T20:33:19.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.408+0000 7f7902930700 1 -- 172.21.15.181:0/2919826569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78fc102d60 msgr2=0x7f78fc19fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:19.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.408+0000 7f7902930700 1 --2- 172.21.15.181:0/2919826569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78fc102d60 0x7f78fc19fcd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f78f0000c00 tx=0x7f78f00047b0 comp rx=0 tx=0).stop 2024-05-03T20:33:19.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.408+0000 7f7902930700 1 -- 172.21.15.181:0/2919826569 shutdown_connections 2024-05-03T20:33:19.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.408+0000 7f7902930700 1 --2- 172.21.15.181:0/2919826569 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f78e40b2880 0x7f78e40b4d40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:19.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.408+0000 7f7902930700 1 --2- 172.21.15.181:0/2919826569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78fc108410 0x7f78fc199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:19.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.408+0000 7f7902930700 1 --2- 172.21.15.181:0/2919826569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78fc1023a0 0x7f78fc19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:19.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.408+0000 7f7902930700 1 --2- 172.21.15.181:0/2919826569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78fc102d60 0x7f78fc19fcd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:19.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.408+0000 7f7902930700 1 -- 172.21.15.181:0/2919826569 >> 172.21.15.181:0/2919826569 conn(0x7f78fc079350 msgr2=0x7f78fc101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:19.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.408+0000 7f7902930700 1 -- 172.21.15.181:0/2919826569 shutdown_connections 2024-05-03T20:33:19.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:19.408+0000 7f7902930700 1 -- 172.21.15.181:0/2919826569 wait complete. 2024-05-03T20:33:19.407 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 10 2024-05-03T20:33:19.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:19 smithi181 ceph-mon[29801]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:19.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:19 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:19.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:19 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:19.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:19 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:19.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:19 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:33:19.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:19 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:33:19.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:19 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:19.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:19 smithi181 ceph-mon[29801]: Deploying daemon mds.g on smithi181 2024-05-03T20:33:19.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:19 smithi181 ceph-mon[29801]: mds.? [v2:172.21.15.204:6838/1644828559,v1:172.21.15.204:6839/1644828559] up:boot 2024-05-03T20:33:19.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:19 smithi181 ceph-mon[29801]: fsmap 9 up:standby 2024-05-03T20:33:19.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:19 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T20:33:19.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:19 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/2919826569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:33:19.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:19 smithi196 ceph-mon[32858]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:19.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:19 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:19.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:19 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:19.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:19 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:19.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:19 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:33:19.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:19 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:33:19.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:19 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:19.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:19 smithi196 ceph-mon[32858]: Deploying daemon mds.g on smithi181 2024-05-03T20:33:19.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:19 smithi196 ceph-mon[32858]: mds.? [v2:172.21.15.204:6838/1644828559,v1:172.21.15.204:6839/1644828559] up:boot 2024-05-03T20:33:19.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:19 smithi196 ceph-mon[32858]: fsmap 9 up:standby 2024-05-03T20:33:19.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:19 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T20:33:19.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:19 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/2919826569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:33:19.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:19 smithi204 ceph-mon[33595]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:19.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:19.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:19.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:19.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:33:19.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:33:19.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:19.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:19 smithi204 ceph-mon[33595]: Deploying daemon mds.g on smithi181 2024-05-03T20:33:19.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:19 smithi204 ceph-mon[33595]: mds.? [v2:172.21.15.204:6838/1644828559,v1:172.21.15.204:6839/1644828559] up:boot 2024-05-03T20:33:19.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:19 smithi204 ceph-mon[33595]: fsmap 9 up:standby 2024-05-03T20:33:19.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T20:33:19.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:19 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/2919826569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:33:21.762 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:21 smithi181 ceph-mon[29801]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:21.763 INFO:journalctl@ceph.mds.g.smithi181.stdout:May 03 20:33:21 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-g[71911]: starting mds.g at 2024-05-03T20:33:21.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:21 smithi196 ceph-mon[32858]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:21.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:21 smithi204 ceph-mon[33595]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:22.039 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T20:33:22.061 INFO:journalctl@ceph.mds.g.smithi181.stdout:May 03 20:33:21 smithi181 bash[71896]: d37b47955bc7e24e67ee3370ccde96dbd9fc95958c755eaa4e675fc0fc2fe912 2024-05-03T20:33:22.061 INFO:journalctl@ceph.mds.g.smithi181.stdout:May 03 20:33:21 smithi181 systemd[1]: Started Ceph mds.g for 1cd058ee-098b-11ef-bc97-c7b262605968. 2024-05-03T20:33:22.304 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:33:22.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:22 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:22.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:22 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:22.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:22 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:22.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:22 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:33:22.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:22 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:33:22.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:22 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:22.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:22 smithi181 ceph-mon[29801]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:22.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:22 smithi181 ceph-mon[29801]: Deploying daemon mds.a on smithi181 2024-05-03T20:33:22.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:22 smithi181 ceph-mon[29801]: mds.? [v2:172.21.15.181:6838/257361276,v1:172.21.15.181:6839/257361276] up:boot 2024-05-03T20:33:22.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:22 smithi181 ceph-mon[29801]: fsmap 10 up:standby 2024-05-03T20:33:22.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:22 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-03T20:33:23.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:22 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:23.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:22 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:23.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:22 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:23.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:22 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:33:23.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:22 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:33:23.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:22 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:23.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:22 smithi196 ceph-mon[32858]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:23.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:22 smithi196 ceph-mon[32858]: Deploying daemon mds.a on smithi181 2024-05-03T20:33:23.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:22 smithi196 ceph-mon[32858]: mds.? [v2:172.21.15.181:6838/257361276,v1:172.21.15.181:6839/257361276] up:boot 2024-05-03T20:33:23.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:22 smithi196 ceph-mon[32858]: fsmap 10 up:standby 2024-05-03T20:33:23.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:22 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-03T20:33:23.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:22 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:23.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:22 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:23.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:22 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:23.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:22 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:33:23.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:22 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:33:23.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:22 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:23.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:22 smithi204 ceph-mon[33595]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:23.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:22 smithi204 ceph-mon[33595]: Deploying daemon mds.a on smithi181 2024-05-03T20:33:23.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:22 smithi204 ceph-mon[33595]: mds.? [v2:172.21.15.181:6838/257361276,v1:172.21.15.181:6839/257361276] up:boot 2024-05-03T20:33:23.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:22 smithi204 ceph-mon[33595]: fsmap 10 up:standby 2024-05-03T20:33:23.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:22 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-03T20:33:24.930 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:24 smithi181 ceph-mon[29801]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:24.932 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.931+0000 7fad8a07f700 1 -- 172.21.15.181:0/2508534723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad8406c3d0 msgr2=0x7fad84110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:24.932 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.931+0000 7fad8a07f700 1 --2- 172.21.15.181:0/2508534723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad8406c3d0 0x7fad84110320 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00b0a0 tx=0x7fad7c00b3b0 comp rx=0 tx=0).stop 2024-05-03T20:33:24.932 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.931+0000 7fad8a07f700 1 -- 172.21.15.181:0/2508534723 shutdown_connections 2024-05-03T20:33:24.932 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.931+0000 7fad8a07f700 1 --2- 172.21.15.181:0/2508534723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad8406c3d0 0x7fad84110320 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:24.932 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.931+0000 7fad8a07f700 1 --2- 172.21.15.181:0/2508534723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad84109500 0x7fad841098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:24.932 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.931+0000 7fad8a07f700 1 --2- 172.21.15.181:0/2508534723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad8406ba10 0x7fad8406be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:24.932 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.931+0000 7fad8a07f700 1 -- 172.21.15.181:0/2508534723 >> 172.21.15.181:0/2508534723 conn(0x7fad84079310 msgr2=0x7fad84079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:24.932 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.931+0000 7fad8a07f700 1 -- 172.21.15.181:0/2508534723 shutdown_connections 2024-05-03T20:33:24.932 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.931+0000 7fad8a07f700 1 -- 172.21.15.181:0/2508534723 wait complete. 2024-05-03T20:33:24.932 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.932+0000 7fad8a07f700 1 Processor -- start 2024-05-03T20:33:24.932 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.932+0000 7fad8a07f700 1 -- start start 2024-05-03T20:33:24.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.932+0000 7fad8a07f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad8406ba10 0x7fad8419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:24.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.932+0000 7fad8a07f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad8406c3d0 0x7fad8419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:24.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.932+0000 7fad8a07f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad84109500 0x7fad84199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:24.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.932+0000 7fad8a07f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fad84113630 con 0x7fad8406c3d0 2024-05-03T20:33:24.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.932+0000 7fad8a07f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fad841134b0 con 0x7fad8406ba10 2024-05-03T20:33:24.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.932+0000 7fad8a07f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fad841137b0 con 0x7fad84109500 2024-05-03T20:33:24.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.932+0000 7fad82ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad8406c3d0 0x7fad8419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:24.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.932+0000 7fad82ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad8406c3d0 0x7fad8419fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39724/0 (socket says 172.21.15.181:39724) 2024-05-03T20:33:24.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.932+0000 7fad82ffd700 1 -- 172.21.15.181:0/1279557725 learned_addr learned my addr 172.21.15.181:0/1279557725 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:33:24.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.933+0000 7fad837fe700 1 --2- 172.21.15.181:0/1279557725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad8406ba10 0x7fad8419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:24.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.933+0000 7fad83fff700 1 --2- 172.21.15.181:0/1279557725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad84109500 0x7fad84199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:24.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.933+0000 7fad82ffd700 1 -- 172.21.15.181:0/1279557725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad84109500 msgr2=0x7fad84199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:24.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.933+0000 7fad82ffd700 1 --2- 172.21.15.181:0/1279557725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad84109500 0x7fad84199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:24.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.933+0000 7fad82ffd700 1 -- 172.21.15.181:0/1279557725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad8406ba10 msgr2=0x7fad8419f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:24.934 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.933+0000 7fad82ffd700 1 --2- 172.21.15.181:0/1279557725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad8406ba10 0x7fad8419f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:24.934 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.933+0000 7fad82ffd700 1 -- 172.21.15.181:0/1279557725 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8419a0d0 con 0x7fad8406c3d0 2024-05-03T20:33:24.934 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.933+0000 7fad837fe700 1 --2- 172.21.15.181:0/1279557725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad8406ba10 0x7fad8419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:33:24.934 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.933+0000 7fad82ffd700 1 --2- 172.21.15.181:0/1279557725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad8406c3d0 0x7fad8419fca0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fad7800d980 tx=0x7fad7800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:24.934 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.933+0000 7fad80ff9700 1 -- 172.21.15.181:0/1279557725 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad78017070 con 0x7fad8406c3d0 2024-05-03T20:33:24.934 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.933+0000 7fad80ff9700 1 -- 172.21.15.181:0/1279557725 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad78004830 con 0x7fad8406c3d0 2024-05-03T20:33:24.934 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.933+0000 7fad80ff9700 1 -- 172.21.15.181:0/1279557725 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad78005440 con 0x7fad8406c3d0 2024-05-03T20:33:24.934 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.933+0000 7fad8a07f700 1 -- 172.21.15.181:0/1279557725 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad8419a3c0 con 0x7fad8406c3d0 2024-05-03T20:33:24.934 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.933+0000 7fad8a07f700 1 -- 172.21.15.181:0/1279557725 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad841a6630 con 0x7fad8406c3d0 2024-05-03T20:33:24.935 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.934+0000 7fad8a07f700 1 -- 172.21.15.181:0/1279557725 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad8406a190 con 0x7fad8406c3d0 2024-05-03T20:33:24.935 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.936+0000 7fad80ff9700 1 -- 172.21.15.181:0/1279557725 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fad780055e0 con 0x7fad8406c3d0 2024-05-03T20:33:24.935 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.937+0000 7fad80ff9700 1 --2- 172.21.15.181:0/1279557725 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fad6c0b2880 0x7fad6c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:24.935 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.937+0000 7fad80ff9700 1 -- 172.21.15.181:0/1279557725 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fad780e20f0 con 0x7fad8406c3d0 2024-05-03T20:33:24.936 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.937+0000 7fad837fe700 1 --2- 172.21.15.181:0/1279557725 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fad6c0b2880 0x7fad6c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:24.936 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.938+0000 7fad837fe700 1 --2- 172.21.15.181:0/1279557725 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fad6c0b2880 0x7fad6c0b4d40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fad74005ea0 tx=0x7fad74005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:24.939 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:24.940+0000 7fad80ff9700 1 -- 172.21.15.181:0/1279557725 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fad78020020 con 0x7fad8406c3d0 2024-05-03T20:33:25.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:25.206+0000 7fad8a07f700 1 -- 172.21.15.181:0/1279557725 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad8402e510 con 0x7fad8406c3d0 2024-05-03T20:33:25.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:25.207+0000 7fad80ff9700 1 -- 172.21.15.181:0/1279557725 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8845 (secure 0 0 0) 0x7fad780aa2b0 con 0x7fad8406c3d0 2024-05-03T20:33:25.206 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:33:25.206 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2414172454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2414172454},{"type":"v1","addr":"172.21.15.181:6835","nonce":2414172454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/204127189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":204127189},{"type":"v1","addr":"172.21.15.181:6837","nonce":204127189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/257361276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":257361276},{"type":"v1","addr":"172.21.15.181:6839","nonce":257361276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3975688022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3975688022},{"type":"v1","addr":"172.21.15.196:6833","nonce":3975688022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2409628823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2409628823},{"type":"v1","addr":"172.21.15.204:6833","nonce":2409628823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1820206106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1820206106},{"type":"v1","addr":"172.21.15.196:6835","nonce":1820206106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3870850287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3870850287},{"type":"v1","addr":"172.21.15.204:6835","nonce":3870850287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3006728559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3006728559},{"type":"v1","addr":"172.21.15.204:6837","nonce":3006728559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/1529420259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1529420259},{"type":"v1","addr":"172.21.15.196:6837","nonce":1529420259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/1644828559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1644828559},{"type":"v1","addr":"172.21.15.204:6839","nonce":1644828559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-03T20:33:25.209 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:25.210+0000 7fad8a07f700 1 -- 172.21.15.181:0/1279557725 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fad6c0b2880 msgr2=0x7fad6c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:25.209 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:25.210+0000 7fad8a07f700 1 --2- 172.21.15.181:0/1279557725 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fad6c0b2880 0x7fad6c0b4d40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fad74005ea0 tx=0x7fad74005e30 comp rx=0 tx=0).stop 2024-05-03T20:33:25.209 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:25.210+0000 7fad8a07f700 1 -- 172.21.15.181:0/1279557725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad8406c3d0 msgr2=0x7fad8419fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:25.209 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:25.210+0000 7fad8a07f700 1 --2- 172.21.15.181:0/1279557725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad8406c3d0 0x7fad8419fca0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fad7800d980 tx=0x7fad7800dd40 comp rx=0 tx=0).stop 2024-05-03T20:33:25.209 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:25.210+0000 7fad8a07f700 1 -- 172.21.15.181:0/1279557725 shutdown_connections 2024-05-03T20:33:25.209 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:25.210+0000 7fad8a07f700 1 --2- 172.21.15.181:0/1279557725 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fad6c0b2880 0x7fad6c0b4d40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:25.209 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:25.210+0000 7fad8a07f700 1 --2- 172.21.15.181:0/1279557725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad84109500 0x7fad84199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:25.209 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:25.210+0000 7fad8a07f700 1 --2- 172.21.15.181:0/1279557725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad8406ba10 0x7fad8419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:25.209 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:25.210+0000 7fad8a07f700 1 --2- 172.21.15.181:0/1279557725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad8406c3d0 0x7fad8419fca0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:25.209 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:25.210+0000 7fad8a07f700 1 -- 172.21.15.181:0/1279557725 >> 172.21.15.181:0/1279557725 conn(0x7fad84079310 msgr2=0x7fad84101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:25.209 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:25.210+0000 7fad8a07f700 1 -- 172.21.15.181:0/1279557725 shutdown_connections 2024-05-03T20:33:25.209 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:25.210+0000 7fad8a07f700 1 -- 172.21.15.181:0/1279557725 wait complete. 2024-05-03T20:33:25.210 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 11 2024-05-03T20:33:25.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:24 smithi196 ceph-mon[32858]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:25.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:24 smithi204 ceph-mon[33595]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:26.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:25 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/1279557725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:33:26.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:26.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:26.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:26.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:33:26.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:33:26.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:25 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:26.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:25 smithi181 ceph-mon[29801]: Deploying daemon mds.b on smithi196 2024-05-03T20:33:26.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:25 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/1279557725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:33:26.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:26.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:26.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:26.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:33:26.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:33:26.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:25 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:26.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:25 smithi196 ceph-mon[32858]: Deploying daemon mds.b on smithi196 2024-05-03T20:33:26.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:25 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/1279557725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:33:26.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:26.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:26.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:26.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T20:33:26.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T20:33:26.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:25 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:26.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:25 smithi204 ceph-mon[33595]: Deploying daemon mds.b on smithi196 2024-05-03T20:33:27.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:26 smithi196 ceph-mon[32858]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:27.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:26 smithi196 ceph-mon[32858]: mds.? [v2:172.21.15.181:6840/2403160115,v1:172.21.15.181:6841/2403160115] up:boot 2024-05-03T20:33:27.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:26 smithi196 ceph-mon[32858]: fsmap 11 up:standby 2024-05-03T20:33:27.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:26 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-03T20:33:27.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:26 smithi181 ceph-mon[29801]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:27.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:26 smithi181 ceph-mon[29801]: mds.? [v2:172.21.15.181:6840/2403160115,v1:172.21.15.181:6841/2403160115] up:boot 2024-05-03T20:33:27.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:26 smithi181 ceph-mon[29801]: fsmap 11 up:standby 2024-05-03T20:33:27.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:26 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-03T20:33:27.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:26 smithi204 ceph-mon[33595]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:27.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:26 smithi204 ceph-mon[33595]: mds.? [v2:172.21.15.181:6840/2403160115,v1:172.21.15.181:6841/2403160115] up:boot 2024-05-03T20:33:27.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:26 smithi204 ceph-mon[33595]: fsmap 11 up:standby 2024-05-03T20:33:27.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:26 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-03T20:33:27.857 INFO:journalctl@ceph.mds.b.smithi196.stdout:May 03 20:33:27 smithi196 systemd[1]: Starting Ceph mds.b for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:33:27.974 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T20:33:28.174 INFO:journalctl@ceph.mds.b.smithi196.stdout:May 03 20:33:27 smithi196 podman[63542]: 2024-05-03 20:33:27.803118228 +0000 UTC m=+0.017611336 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T20:33:28.174 INFO:journalctl@ceph.mds.b.smithi196.stdout:May 03 20:33:27 smithi196 podman[63542]: 2024-05-03 20:33:27.928440236 +0000 UTC m=+0.142933326 container create 0cc7fb0361cde5de70797ee63dcdc4ae8ced119b4eb0ef34fbd7df2369eca36a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-03T20:33:28.227 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:33:28.606 INFO:journalctl@ceph.mds.b.smithi196.stdout:May 03 20:33:28 smithi196 podman[63542]: 2024-05-03 20:33:28.176073708 +0000 UTC m=+0.390566800 container init 0cc7fb0361cde5de70797ee63dcdc4ae8ced119b4eb0ef34fbd7df2369eca36a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-03T20:33:28.606 INFO:journalctl@ceph.mds.b.smithi196.stdout:May 03 20:33:28 smithi196 podman[63542]: 2024-05-03 20:33:28.178725044 +0000 UTC m=+0.393218136 container start 0cc7fb0361cde5de70797ee63dcdc4ae8ced119b4eb0ef34fbd7df2369eca36a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-03T20:33:28.607 INFO:journalctl@ceph.mds.b.smithi196.stdout:May 03 20:33:28 smithi196 ceph-mds[63562]: -- 172.21.15.196:0/1351250125 <== mon.2 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556a71e0fa40 con 0x556a71de3000 2024-05-03T20:33:28.607 INFO:journalctl@ceph.mds.b.smithi196.stdout:May 03 20:33:28 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-b[63557]: starting mds.b at 2024-05-03T20:33:28.607 INFO:journalctl@ceph.mds.b.smithi196.stdout:May 03 20:33:28 smithi196 bash[63542]: 0cc7fb0361cde5de70797ee63dcdc4ae8ced119b4eb0ef34fbd7df2369eca36a 2024-05-03T20:33:28.607 INFO:journalctl@ceph.mds.b.smithi196.stdout:May 03 20:33:28 smithi196 systemd[1]: Started Ceph mds.b for 1cd058ee-098b-11ef-bc97-c7b262605968. 2024-05-03T20:33:29.539 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:29 smithi181 ceph-mon[29801]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:29.539 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:29 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:29.539 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:29 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:29.539 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:29 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:29.539 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:29 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:29.539 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:29 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:29.539 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:29 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:33:29.539 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:29 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:29.540 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:29 smithi181 ceph-mon[29801]: mds.? [v2:172.21.15.196:6838/1182826743,v1:172.21.15.196:6839/1182826743] up:boot 2024-05-03T20:33:29.540 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:29 smithi181 ceph-mon[29801]: fsmap 12 up:standby 2024-05-03T20:33:29.540 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:29 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T20:33:29.594 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:29 smithi204 ceph-mon[33595]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:29.594 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:29 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:29.594 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:29 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:29.594 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:29 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:29.594 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:29 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:29.594 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:29 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:29.594 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:29 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:33:29.595 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:29 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:29.595 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:29 smithi204 ceph-mon[33595]: mds.? [v2:172.21.15.196:6838/1182826743,v1:172.21.15.196:6839/1182826743] up:boot 2024-05-03T20:33:29.595 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:29 smithi204 ceph-mon[33595]: fsmap 12 up:standby 2024-05-03T20:33:29.595 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:29 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T20:33:29.600 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:29 smithi196 ceph-mon[32858]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:29.600 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:29 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:29.600 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:29 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:29.600 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:29 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:29.600 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:29 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:29.600 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:29 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:29.600 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:29 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:33:29.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:29 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:29.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:29 smithi196 ceph-mon[32858]: mds.? [v2:172.21.15.196:6838/1182826743,v1:172.21.15.196:6839/1182826743] up:boot 2024-05-03T20:33:29.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:29 smithi196 ceph-mon[32858]: fsmap 12 up:standby 2024-05-03T20:33:29.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:29 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T20:33:29.737 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.737+0000 7f7cc4f6f700 1 -- 172.21.15.181:0/527067453 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc003c60 con 0x7f7cc0108850 2024-05-03T20:33:29.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.738+0000 7f7cc81d5700 1 -- 172.21.15.181:0/527067453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cc0108850 msgr2=0x7f7cc01103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:29.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.738+0000 7f7cc81d5700 1 --2- 172.21.15.181:0/527067453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cc0108850 0x7f7cc01103d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc0099c0 tx=0x7f7cbc009cd0 comp rx=0 tx=0).stop 2024-05-03T20:33:29.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.738+0000 7f7cc81d5700 1 -- 172.21.15.181:0/527067453 shutdown_connections 2024-05-03T20:33:29.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.738+0000 7f7cc81d5700 1 --2- 172.21.15.181:0/527067453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cc0108850 0x7f7cc01103d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:29.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.738+0000 7f7cc81d5700 1 --2- 172.21.15.181:0/527067453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cc0103750 0x7f7cc0103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:29.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.738+0000 7f7cc81d5700 1 --2- 172.21.15.181:0/527067453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cc0104100 0x7f7cc0108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:29.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.738+0000 7f7cc81d5700 1 -- 172.21.15.181:0/527067453 >> 172.21.15.181:0/527067453 conn(0x7f7cc007a600 msgr2=0x7f7cc007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:29.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.738+0000 7f7cc81d5700 1 -- 172.21.15.181:0/527067453 shutdown_connections 2024-05-03T20:33:29.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.738+0000 7f7cc81d5700 1 -- 172.21.15.181:0/527067453 wait complete. 2024-05-03T20:33:29.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.739+0000 7f7cc81d5700 1 Processor -- start 2024-05-03T20:33:29.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.739+0000 7f7cc81d5700 1 -- start start 2024-05-03T20:33:29.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.739+0000 7f7cc81d5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cc0103750 0x7f7cc019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:29.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.739+0000 7f7cc81d5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cc0104100 0x7f7cc019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:29.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.739+0000 7f7cc81d5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cc0108850 0x7f7cc01998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:29.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.739+0000 7f7cc81d5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc0113730 con 0x7f7cc0104100 2024-05-03T20:33:29.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.739+0000 7f7cc81d5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc01135b0 con 0x7f7cc0103750 2024-05-03T20:33:29.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.739+0000 7f7cc81d5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc01138b0 con 0x7f7cc0108850 2024-05-03T20:33:29.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.739+0000 7f7cc5770700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cc0104100 0x7f7cc019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:29.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.739+0000 7f7cc5770700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cc0104100 0x7f7cc019fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40970/0 (socket says 172.21.15.181:40970) 2024-05-03T20:33:29.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.739+0000 7f7cc5f71700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cc0103750 0x7f7cc019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:29.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.739+0000 7f7cc5770700 1 -- 172.21.15.181:0/1808466070 learned_addr learned my addr 172.21.15.181:0/1808466070 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:33:29.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.739+0000 7f7cc6772700 1 --2- 172.21.15.181:0/1808466070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cc0108850 0x7f7cc01998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:29.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.740+0000 7f7cc5770700 1 -- 172.21.15.181:0/1808466070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cc0108850 msgr2=0x7f7cc01998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:29.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.740+0000 7f7cc5770700 1 --2- 172.21.15.181:0/1808466070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cc0108850 0x7f7cc01998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:29.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.740+0000 7f7cc5770700 1 -- 172.21.15.181:0/1808466070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cc0103750 msgr2=0x7f7cc019f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:29.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.740+0000 7f7cc5770700 1 --2- 172.21.15.181:0/1808466070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cc0103750 0x7f7cc019f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:29.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.740+0000 7f7cc5770700 1 -- 172.21.15.181:0/1808466070 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc019a020 con 0x7f7cc0104100 2024-05-03T20:33:29.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.740+0000 7f7cc5770700 1 --2- 172.21.15.181:0/1808466070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cc0104100 0x7f7cc019fc40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7cb000b530 tx=0x7f7cb000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:29.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.740+0000 7f7cb6ffd700 1 -- 172.21.15.181:0/1808466070 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0015020 con 0x7f7cc0104100 2024-05-03T20:33:29.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.740+0000 7f7cb6ffd700 1 -- 172.21.15.181:0/1808466070 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cb0011ce0 con 0x7f7cc0104100 2024-05-03T20:33:29.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.740+0000 7f7cc81d5700 1 -- 172.21.15.181:0/1808466070 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc019a310 con 0x7f7cc0104100 2024-05-03T20:33:29.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.740+0000 7f7cb6ffd700 1 -- 172.21.15.181:0/1808466070 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0010dd0 con 0x7f7cc0104100 2024-05-03T20:33:29.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.740+0000 7f7cc81d5700 1 -- 172.21.15.181:0/1808466070 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc00756d0 con 0x7f7cc0104100 2024-05-03T20:33:29.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.742+0000 7f7cc81d5700 1 -- 172.21.15.181:0/1808466070 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cc006a190 con 0x7f7cc0104100 2024-05-03T20:33:29.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.743+0000 7f7cb6ffd700 1 -- 172.21.15.181:0/1808466070 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f7cb0018020 con 0x7f7cc0104100 2024-05-03T20:33:29.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.743+0000 7f7cb6ffd700 1 --2- 172.21.15.181:0/1808466070 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f7cac0b2900 0x7f7cac0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:29.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.744+0000 7f7cc5f71700 1 --2- 172.21.15.181:0/1808466070 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f7cac0b2900 0x7f7cac0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:29.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.744+0000 7f7cb6ffd700 1 -- 172.21.15.181:0/1808466070 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f7cb00e1420 con 0x7f7cc0104100 2024-05-03T20:33:29.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.744+0000 7f7cc5f71700 1 --2- 172.21.15.181:0/1808466070 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f7cac0b2900 0x7f7cac0b4dc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8009550 tx=0x7f7cb8013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:29.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:29.748+0000 7f7cb6ffd700 1 -- 172.21.15.181:0/1808466070 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7cb001f020 con 0x7f7cc0104100 2024-05-03T20:33:30.012 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:30.012+0000 7f7cc81d5700 1 -- 172.21.15.181:0/1808466070 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7cc002e510 con 0x7f7cc0104100 2024-05-03T20:33:30.013 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:30.013+0000 7f7cb6ffd700 1 -- 172.21.15.181:0/1808466070 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10503 (secure 0 0 0) 0x7f7cb00a9540 con 0x7f7cc0104100 2024-05-03T20:33:30.013 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:33:30.013 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2414172454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2414172454},{"type":"v1","addr":"172.21.15.181:6835","nonce":2414172454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/204127189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":204127189},{"type":"v1","addr":"172.21.15.181:6837","nonce":204127189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/257361276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":257361276},{"type":"v1","addr":"172.21.15.181:6839","nonce":257361276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6841/2403160115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2403160115},{"type":"v1","addr":"172.21.15.181:6841","nonce":2403160115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3975688022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3975688022},{"type":"v1","addr":"172.21.15.196:6833","nonce":3975688022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2409628823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2409628823},{"type":"v1","addr":"172.21.15.204:6833","nonce":2409628823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1820206106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1820206106},{"type":"v1","addr":"172.21.15.196:6835","nonce":1820206106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3870850287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3870850287},{"type":"v1","addr":"172.21.15.204:6835","nonce":3870850287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3006728559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3006728559},{"type":"v1","addr":"172.21.15.204:6837","nonce":3006728559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/1529420259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1529420259},{"type":"v1","addr":"172.21.15.196:6837","nonce":1529420259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/1644828559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1644828559},{"type":"v1","addr":"172.21.15.204:6839","nonce":1644828559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24488,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/1182826743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1182826743},{"type":"v1","addr":"172.21.15.196:6839","nonce":1182826743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-03T20:33:30.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:30.015+0000 7f7cc81d5700 1 -- 172.21.15.181:0/1808466070 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f7cac0b2900 msgr2=0x7f7cac0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:30.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:30.015+0000 7f7cc81d5700 1 --2- 172.21.15.181:0/1808466070 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f7cac0b2900 0x7f7cac0b4dc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8009550 tx=0x7f7cb8013040 comp rx=0 tx=0).stop 2024-05-03T20:33:30.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:30.015+0000 7f7cc81d5700 1 -- 172.21.15.181:0/1808466070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cc0104100 msgr2=0x7f7cc019fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:30.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:30.015+0000 7f7cc81d5700 1 --2- 172.21.15.181:0/1808466070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cc0104100 0x7f7cc019fc40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7cb000b530 tx=0x7f7cb000b840 comp rx=0 tx=0).stop 2024-05-03T20:33:30.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:30.015+0000 7f7cc81d5700 1 -- 172.21.15.181:0/1808466070 shutdown_connections 2024-05-03T20:33:30.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:30.015+0000 7f7cc81d5700 1 --2- 172.21.15.181:0/1808466070 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f7cac0b2900 0x7f7cac0b4dc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:30.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:30.015+0000 7f7cc81d5700 1 --2- 172.21.15.181:0/1808466070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cc0108850 0x7f7cc01998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:30.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:30.015+0000 7f7cc81d5700 1 --2- 172.21.15.181:0/1808466070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cc0103750 0x7f7cc019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:30.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:30.015+0000 7f7cc81d5700 1 --2- 172.21.15.181:0/1808466070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cc0104100 0x7f7cc019fc40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:30.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:30.015+0000 7f7cc81d5700 1 -- 172.21.15.181:0/1808466070 >> 172.21.15.181:0/1808466070 conn(0x7f7cc007a600 msgr2=0x7f7cc01014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:30.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:30.015+0000 7f7cc81d5700 1 -- 172.21.15.181:0/1808466070 shutdown_connections 2024-05-03T20:33:30.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:30.015+0000 7f7cc81d5700 1 -- 172.21.15.181:0/1808466070 wait complete. 2024-05-03T20:33:30.016 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 13 2024-05-03T20:33:30.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:30 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/1808466070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:33:30.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:30 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:30.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:30 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:30.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:30 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:30.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:30 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:30.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:30 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/1808466070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:33:30.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:30 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:30.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:30 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:30.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:30 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:30.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:30 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:30.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:30 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/1808466070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:33:30.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:30 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:30.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:30 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:30.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:30 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:30.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:30 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:30.748 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-03T20:33:31.176 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:33:31.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:31 smithi196 ceph-mon[32858]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:31.609 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:31 smithi181 ceph-mon[29801]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:31.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:31 smithi204 ceph-mon[33595]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:32.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.792+0000 7fdcad262700 1 -- 172.21.15.181:0/3462296047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdca8103750 msgr2=0x7fdca8103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:32.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.792+0000 7fdcad262700 1 --2- 172.21.15.181:0/3462296047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdca8103750 0x7fdca8103b30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdca000b0d0 tx=0x7fdca000b3e0 comp rx=0 tx=0).stop 2024-05-03T20:33:32.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.792+0000 7fdcad262700 1 -- 172.21.15.181:0/3462296047 shutdown_connections 2024-05-03T20:33:32.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.792+0000 7fdcad262700 1 --2- 172.21.15.181:0/3462296047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdca8108850 0x7fdca81103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:32.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.792+0000 7fdcad262700 1 --2- 172.21.15.181:0/3462296047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdca8103750 0x7fdca8103b30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:32.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.792+0000 7fdcad262700 1 --2- 172.21.15.181:0/3462296047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdca8104100 0x7fdca8108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:32.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.792+0000 7fdcad262700 1 -- 172.21.15.181:0/3462296047 >> 172.21.15.181:0/3462296047 conn(0x7fdca807a600 msgr2=0x7fdca807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:32.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.792+0000 7fdcad262700 1 -- 172.21.15.181:0/3462296047 shutdown_connections 2024-05-03T20:33:32.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.792+0000 7fdcad262700 1 -- 172.21.15.181:0/3462296047 wait complete. 2024-05-03T20:33:32.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.793+0000 7fdcad262700 1 Processor -- start 2024-05-03T20:33:32.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.793+0000 7fdcad262700 1 -- start start 2024-05-03T20:33:32.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.793+0000 7fdcad262700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdca8103750 0x7fdca819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:32.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.793+0000 7fdcad262700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdca8104100 0x7fdca819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:32.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.793+0000 7fdcad262700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdca8108850 0x7fdca8199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:32.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.793+0000 7fdcad262700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdca81136a0 con 0x7fdca8108850 2024-05-03T20:33:32.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.793+0000 7fdcad262700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdca8113520 con 0x7fdca8103750 2024-05-03T20:33:32.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.793+0000 7fdcad262700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdca8113820 con 0x7fdca8104100 2024-05-03T20:33:32.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.793+0000 7fdca77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdca8108850 0x7fdca8199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:32.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.793+0000 7fdca77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdca8108850 0x7fdca8199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40980/0 (socket says 172.21.15.181:40980) 2024-05-03T20:33:32.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.793+0000 7fdca77fe700 1 -- 172.21.15.181:0/3948766412 learned_addr learned my addr 172.21.15.181:0/3948766412 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:33:32.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.793+0000 7fdca6ffd700 1 --2- 172.21.15.181:0/3948766412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdca8103750 0x7fdca819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:32.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.794+0000 7fdca67fc700 1 --2- 172.21.15.181:0/3948766412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdca8104100 0x7fdca819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:32.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.794+0000 7fdca77fe700 1 -- 172.21.15.181:0/3948766412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdca8104100 msgr2=0x7fdca819fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:32.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.794+0000 7fdca77fe700 1 --2- 172.21.15.181:0/3948766412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdca8104100 0x7fdca819fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:32.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.794+0000 7fdca77fe700 1 -- 172.21.15.181:0/3948766412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdca8103750 msgr2=0x7fdca819f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:32.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.794+0000 7fdca77fe700 1 --2- 172.21.15.181:0/3948766412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdca8103750 0x7fdca819f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:32.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.794+0000 7fdca77fe700 1 -- 172.21.15.181:0/3948766412 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca819a160 con 0x7fdca8108850 2024-05-03T20:33:32.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.794+0000 7fdca77fe700 1 --2- 172.21.15.181:0/3948766412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdca8108850 0x7fdca8199a00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800b550 tx=0x7fdc9800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:32.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.794+0000 7fdc9ffff700 1 -- 172.21.15.181:0/3948766412 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc98015020 con 0x7fdca8108850 2024-05-03T20:33:32.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.794+0000 7fdcad262700 1 -- 172.21.15.181:0/3948766412 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca819a450 con 0x7fdca8108850 2024-05-03T20:33:32.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.794+0000 7fdc9ffff700 1 -- 172.21.15.181:0/3948766412 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc98011ce0 con 0x7fdca8108850 2024-05-03T20:33:32.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.794+0000 7fdc9ffff700 1 -- 172.21.15.181:0/3948766412 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc98010dd0 con 0x7fdca8108850 2024-05-03T20:33:32.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.794+0000 7fdcad262700 1 -- 172.21.15.181:0/3948766412 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca81a66c0 con 0x7fdca8108850 2024-05-03T20:33:32.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.795+0000 7fdcad262700 1 -- 172.21.15.181:0/3948766412 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdca810ba90 con 0x7fdca8108850 2024-05-03T20:33:32.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.796+0000 7fdc9ffff700 1 -- 172.21.15.181:0/3948766412 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fdc98018020 con 0x7fdca8108850 2024-05-03T20:33:32.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.797+0000 7fdc9ffff700 1 --2- 172.21.15.181:0/3948766412 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fdc940b2880 0x7fdc940b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:32.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.797+0000 7fdca6ffd700 1 --2- 172.21.15.181:0/3948766412 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fdc940b2880 0x7fdc940b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:32.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.797+0000 7fdc9ffff700 1 -- 172.21.15.181:0/3948766412 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fdc980e1a10 con 0x7fdca8108850 2024-05-03T20:33:32.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.797+0000 7fdca6ffd700 1 --2- 172.21.15.181:0/3948766412 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fdc940b2880 0x7fdc940b4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdca0009fd0 tx=0x7fdca000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:32.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:32.800+0000 7fdc9ffff700 1 -- 172.21.15.181:0/3948766412 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fdc9801f020 con 0x7fdca8108850 2024-05-03T20:33:33.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:33.025+0000 7fdcad262700 1 -- 172.21.15.181:0/3948766412 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fdca819aee0 con 0x7fdca8108850 2024-05-03T20:33:33.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:33.026+0000 7fdc9ffff700 1 -- 172.21.15.181:0/3948766412 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v74) ==== 84+0+65 (secure 0 0 0) 0x7fdc980a9bd0 con 0x7fdca8108850 2024-05-03T20:33:33.026 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:33:33.026 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2024-05-03T20:33:33.026 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-05-03T20:33:33.026 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2024-05-03T20:33:33.026 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": ".mgr" 2024-05-03T20:33:33.026 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-03T20:33:33.026 INFO:teuthology.orchestra.run.smithi181.stdout:] 2024-05-03T20:33:33.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:33.028+0000 7fdcad262700 1 -- 172.21.15.181:0/3948766412 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fdc940b2880 msgr2=0x7fdc940b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:33.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:33.029+0000 7fdcad262700 1 --2- 172.21.15.181:0/3948766412 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fdc940b2880 0x7fdc940b4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdca0009fd0 tx=0x7fdca000b040 comp rx=0 tx=0).stop 2024-05-03T20:33:33.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:33.029+0000 7fdcad262700 1 -- 172.21.15.181:0/3948766412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdca8108850 msgr2=0x7fdca8199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:33.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:33.029+0000 7fdcad262700 1 --2- 172.21.15.181:0/3948766412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdca8108850 0x7fdca8199a00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800b550 tx=0x7fdc9800b860 comp rx=0 tx=0).stop 2024-05-03T20:33:33.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:33.029+0000 7fdcad262700 1 -- 172.21.15.181:0/3948766412 shutdown_connections 2024-05-03T20:33:33.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:33.029+0000 7fdcad262700 1 --2- 172.21.15.181:0/3948766412 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fdc940b2880 0x7fdc940b4d40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:33.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:33.029+0000 7fdcad262700 1 --2- 172.21.15.181:0/3948766412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdca8108850 0x7fdca8199a00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:33.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:33.029+0000 7fdcad262700 1 --2- 172.21.15.181:0/3948766412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdca8103750 0x7fdca819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:33.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:33.029+0000 7fdcad262700 1 --2- 172.21.15.181:0/3948766412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdca8104100 0x7fdca819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:33.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:33.029+0000 7fdcad262700 1 -- 172.21.15.181:0/3948766412 >> 172.21.15.181:0/3948766412 conn(0x7fdca807a600 msgr2=0x7fdca81099a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:33.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:33.029+0000 7fdcad262700 1 -- 172.21.15.181:0/3948766412 shutdown_connections 2024-05-03T20:33:33.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:33.029+0000 7fdcad262700 1 -- 172.21.15.181:0/3948766412 wait complete. 2024-05-03T20:33:33.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:33 smithi204 ceph-mon[33595]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:33.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:33 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3948766412' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-03T20:33:33.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:33 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:33.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:33 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:33.654 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:33 smithi181 ceph-mon[29801]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:33.654 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:33 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3948766412' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-03T20:33:33.654 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:33 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:33.654 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:33 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:33.680 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-03T20:33:33.680 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-03T20:33:33.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:33 smithi196 ceph-mon[32858]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:33.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:33 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3948766412' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-03T20:33:33.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:33 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:33.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:33 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:33.953 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:33:34.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:34 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:34.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:34 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:33:34.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:34 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:34.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:34 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:33:34.669 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:34 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:34.669 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:34 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:33:34.669 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:34 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:34.669 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:34 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:33:34.802 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:34 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:34.802 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:34 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:33:34.802 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:34 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:34.802 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:34 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:33:35.505 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.501+0000 7f28bafd4700 1 -- 172.21.15.181:0/3888658509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28b4103080 msgr2=0x7f28b410f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:35.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.501+0000 7f28bafd4700 1 --2- 172.21.15.181:0/3888658509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28b4103080 0x7f28b410f1f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f289c009990 tx=0x7f289c009ca0 comp rx=0 tx=0).stop 2024-05-03T20:33:35.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.501+0000 7f28bafd4700 1 -- 172.21.15.181:0/3888658509 shutdown_connections 2024-05-03T20:33:35.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.501+0000 7f28bafd4700 1 --2- 172.21.15.181:0/3888658509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28b4103080 0x7f28b410f1f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:35.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.501+0000 7f28bafd4700 1 --2- 172.21.15.181:0/3888658509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28b4108510 0x7f28b41088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:35.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.501+0000 7f28bafd4700 1 --2- 172.21.15.181:0/3888658509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28b41026c0 0x7f28b4102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:35.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.501+0000 7f28bafd4700 1 -- 172.21.15.181:0/3888658509 >> 172.21.15.181:0/3888658509 conn(0x7f28b4079100 msgr2=0x7f28b4079510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:35.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.501+0000 7f28bafd4700 1 -- 172.21.15.181:0/3888658509 shutdown_connections 2024-05-03T20:33:35.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.501+0000 7f28bafd4700 1 -- 172.21.15.181:0/3888658509 wait complete. 2024-05-03T20:33:35.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.501+0000 7f28bafd4700 1 Processor -- start 2024-05-03T20:33:35.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.502+0000 7f28bafd4700 1 -- start start 2024-05-03T20:33:35.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.502+0000 7f28bafd4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28b41026c0 0x7f28b419e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:35.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.502+0000 7f28bafd4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28b4103080 0x7f28b419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:35.507 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.502+0000 7f28bafd4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28b4108510 0x7f28b4199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:35.507 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.502+0000 7f28bafd4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f28b41125a0 con 0x7f28b41026c0 2024-05-03T20:33:35.507 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.502+0000 7f28bafd4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f28b4112420 con 0x7f28b4103080 2024-05-03T20:33:35.507 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.502+0000 7f28bafd4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f28b4112720 con 0x7f28b4108510 2024-05-03T20:33:35.507 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.502+0000 7f28b8d70700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28b41026c0 0x7f28b419e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:35.507 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.502+0000 7f28b8d70700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28b41026c0 0x7f28b419e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35958/0 (socket says 172.21.15.181:35958) 2024-05-03T20:33:35.507 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.502+0000 7f28b8d70700 1 -- 172.21.15.181:0/3386541410 learned_addr learned my addr 172.21.15.181:0/3386541410 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:33:35.507 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.502+0000 7f28b3fff700 1 --2- 172.21.15.181:0/3386541410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28b4103080 0x7f28b419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:35.507 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.502+0000 7f28b9571700 1 --2- 172.21.15.181:0/3386541410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28b4108510 0x7f28b4199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:35.507 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.502+0000 7f28b8d70700 1 -- 172.21.15.181:0/3386541410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28b4108510 msgr2=0x7f28b4199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:35.507 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.502+0000 7f28b8d70700 1 --2- 172.21.15.181:0/3386541410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28b4108510 0x7f28b4199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:35.507 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.502+0000 7f28b8d70700 1 -- 172.21.15.181:0/3386541410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28b4103080 msgr2=0x7f28b419eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:35.507 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.502+0000 7f28b8d70700 1 --2- 172.21.15.181:0/3386541410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28b4103080 0x7f28b419eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:35.508 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.502+0000 7f28b8d70700 1 -- 172.21.15.181:0/3386541410 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b419a3d0 con 0x7f28b41026c0 2024-05-03T20:33:35.508 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.502+0000 7f28b8d70700 1 --2- 172.21.15.181:0/3386541410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28b41026c0 0x7f28b419e660 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f28ac007f10 tx=0x7f28ac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:35.508 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.502+0000 7f28b1ffb700 1 -- 172.21.15.181:0/3386541410 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac004db0 con 0x7f28b41026c0 2024-05-03T20:33:35.508 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.502+0000 7f28b1ffb700 1 -- 172.21.15.181:0/3386541410 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28ac003a00 con 0x7f28b41026c0 2024-05-03T20:33:35.508 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.502+0000 7f28b1ffb700 1 -- 172.21.15.181:0/3386541410 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac0045e0 con 0x7f28b41026c0 2024-05-03T20:33:35.508 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.502+0000 7f28bafd4700 1 -- 172.21.15.181:0/3386541410 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b419a6c0 con 0x7f28b41026c0 2024-05-03T20:33:35.508 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.503+0000 7f28bafd4700 1 -- 172.21.15.181:0/3386541410 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b41a6590 con 0x7f28b41026c0 2024-05-03T20:33:35.508 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.504+0000 7f28bafd4700 1 -- 172.21.15.181:0/3386541410 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b406a190 con 0x7f28b41026c0 2024-05-03T20:33:35.508 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.505+0000 7f28b1ffb700 1 -- 172.21.15.181:0/3386541410 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f28ac01c020 con 0x7f28b41026c0 2024-05-03T20:33:35.508 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.505+0000 7f28b1ffb700 1 --2- 172.21.15.181:0/3386541410 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f28a00b2900 0x7f28a00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:35.508 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.505+0000 7f28b3fff700 1 --2- 172.21.15.181:0/3386541410 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f28a00b2900 0x7f28a00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:35.508 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.505+0000 7f28b1ffb700 1 -- 172.21.15.181:0/3386541410 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f28ac0e1500 con 0x7f28b41026c0 2024-05-03T20:33:35.508 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.506+0000 7f28b3fff700 1 --2- 172.21.15.181:0/3386541410 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f28a00b2900 0x7f28a00b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f28b410f650 tx=0x7f28a400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:35.509 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.510+0000 7f28b1ffb700 1 -- 172.21.15.181:0/3386541410 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f28ac051d70 con 0x7f28b41026c0 2024-05-03T20:33:35.676 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:35 smithi181 ceph-mon[29801]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:35.731 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:35.732+0000 7f28bafd4700 1 -- 172.21.15.181:0/3386541410 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f28b41a67f0 con 0x7f28b41026c0 2024-05-03T20:33:35.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:35 smithi196 ceph-mon[32858]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:35.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:35 smithi204 ceph-mon[33595]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:36.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:36.655+0000 7f28b1ffb700 1 -- 172.21.15.181:0/3386541410 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v75) ==== 138+0+0 (secure 0 0 0) 0x7f28ac010070 con 0x7f28b41026c0 2024-05-03T20:33:36.658 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:36.659+0000 7f28bafd4700 1 -- 172.21.15.181:0/3386541410 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f28a00b2900 msgr2=0x7f28a00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:36.658 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:36.659+0000 7f28bafd4700 1 --2- 172.21.15.181:0/3386541410 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f28a00b2900 0x7f28a00b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f28b410f650 tx=0x7f28a400b410 comp rx=0 tx=0).stop 2024-05-03T20:33:36.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:36.659+0000 7f28bafd4700 1 -- 172.21.15.181:0/3386541410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28b41026c0 msgr2=0x7f28b419e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:36.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:36.659+0000 7f28bafd4700 1 --2- 172.21.15.181:0/3386541410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28b41026c0 0x7f28b419e660 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f28ac007f10 tx=0x7f28ac00d3b0 comp rx=0 tx=0).stop 2024-05-03T20:33:36.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:36.659+0000 7f28bafd4700 1 -- 172.21.15.181:0/3386541410 shutdown_connections 2024-05-03T20:33:36.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:36.659+0000 7f28bafd4700 1 --2- 172.21.15.181:0/3386541410 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f28a00b2900 0x7f28a00b4dc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:36.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:36.659+0000 7f28bafd4700 1 --2- 172.21.15.181:0/3386541410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28b4108510 0x7f28b4199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:36.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:36.659+0000 7f28bafd4700 1 --2- 172.21.15.181:0/3386541410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28b41026c0 0x7f28b419e660 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:36.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:36.659+0000 7f28bafd4700 1 --2- 172.21.15.181:0/3386541410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28b4103080 0x7f28b419eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:36.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:36.659+0000 7f28bafd4700 1 -- 172.21.15.181:0/3386541410 >> 172.21.15.181:0/3386541410 conn(0x7f28b4079100 msgr2=0x7f28b41005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:36.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:36.659+0000 7f28bafd4700 1 -- 172.21.15.181:0/3386541410 shutdown_connections 2024-05-03T20:33:36.660 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:36.659+0000 7f28bafd4700 1 -- 172.21.15.181:0/3386541410 wait complete. 2024-05-03T20:33:36.661 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' created 2024-05-03T20:33:36.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:36.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:36.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:36.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:36.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:36 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3386541410' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-03T20:33:36.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:36.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:36.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:36.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:33:36.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:36 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:36.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:36.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:36.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:36.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:36.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:36 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3386541410' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-03T20:33:36.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:36.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:36.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:36.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:33:36.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:36 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:36.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:36.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:36.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:36.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:36.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:36 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3386541410' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-03T20:33:36.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:36.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:36.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:33:36.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:33:36.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:36 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:33:37.276 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-03T20:33:37.516 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:33:37.890 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:37 smithi181 ceph-mon[29801]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:37.890 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:37 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3386541410' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-03T20:33:37.890 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:37 smithi181 ceph-mon[29801]: osdmap e75: 12 total, 12 up, 12 in 2024-05-03T20:33:38.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:37 smithi196 ceph-mon[32858]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:38.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:37 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3386541410' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-03T20:33:38.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:37 smithi196 ceph-mon[32858]: osdmap e75: 12 total, 12 up, 12 in 2024-05-03T20:33:38.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:37 smithi204 ceph-mon[33595]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:38.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:37 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3386541410' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-03T20:33:38.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:37 smithi204 ceph-mon[33595]: osdmap e75: 12 total, 12 up, 12 in 2024-05-03T20:33:38.883 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:38 smithi181 ceph-mon[29801]: osdmap e76: 12 total, 12 up, 12 in 2024-05-03T20:33:39.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:38 smithi196 ceph-mon[32858]: osdmap e76: 12 total, 12 up, 12 in 2024-05-03T20:33:39.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:38 smithi204 ceph-mon[33595]: osdmap e76: 12 total, 12 up, 12 in 2024-05-03T20:33:39.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.116+0000 7f9720193700 1 -- 172.21.15.181:0/3842216895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97181034f0 msgr2=0x7f9718103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:39.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.116+0000 7f9720193700 1 --2- 172.21.15.181:0/3842216895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97181034f0 0x7f9718103970 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f970c009990 tx=0x7f970c009ca0 comp rx=0 tx=0).stop 2024-05-03T20:33:39.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.116+0000 7f9720193700 1 -- 172.21.15.181:0/3842216895 shutdown_connections 2024-05-03T20:33:39.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.116+0000 7f9720193700 1 --2- 172.21.15.181:0/3842216895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9718103eb0 0x7f9718110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:39.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.116+0000 7f9720193700 1 --2- 172.21.15.181:0/3842216895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9718109600 0x7f97181099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:39.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.116+0000 7f9720193700 1 --2- 172.21.15.181:0/3842216895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97181034f0 0x7f9718103970 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:39.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.116+0000 7f9720193700 1 -- 172.21.15.181:0/3842216895 >> 172.21.15.181:0/3842216895 conn(0x7f971807a650 msgr2=0x7f971807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:39.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.116+0000 7f9720193700 1 -- 172.21.15.181:0/3842216895 shutdown_connections 2024-05-03T20:33:39.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.116+0000 7f9720193700 1 -- 172.21.15.181:0/3842216895 wait complete. 2024-05-03T20:33:39.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.117+0000 7f9720193700 1 Processor -- start 2024-05-03T20:33:39.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.117+0000 7f9720193700 1 -- start start 2024-05-03T20:33:39.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.117+0000 7f9720193700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97181034f0 0x7f971819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:39.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.117+0000 7f9720193700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9718103eb0 0x7f971819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:39.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.117+0000 7f9720193700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9718109600 0x7f9718199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:39.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.117+0000 7f9720193700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9718113670 con 0x7f9718103eb0 2024-05-03T20:33:39.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.117+0000 7f9720193700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f97181134f0 con 0x7f9718109600 2024-05-03T20:33:39.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.117+0000 7f9720193700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f97181137f0 con 0x7f97181034f0 2024-05-03T20:33:39.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.117+0000 7f971d72e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9718103eb0 0x7f971819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:39.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.117+0000 7f971d72e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9718103eb0 0x7f971819fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35984/0 (socket says 172.21.15.181:35984) 2024-05-03T20:33:39.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.117+0000 7f971d72e700 1 -- 172.21.15.181:0/716447650 learned_addr learned my addr 172.21.15.181:0/716447650 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:33:39.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.117+0000 7f971df2f700 1 --2- 172.21.15.181:0/716447650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97181034f0 0x7f971819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:39.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.117+0000 7f971e730700 1 --2- 172.21.15.181:0/716447650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9718109600 0x7f9718199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:39.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.117+0000 7f971d72e700 1 -- 172.21.15.181:0/716447650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97181034f0 msgr2=0x7f971819f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:39.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.117+0000 7f971d72e700 1 --2- 172.21.15.181:0/716447650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97181034f0 0x7f971819f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:39.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.117+0000 7f971d72e700 1 -- 172.21.15.181:0/716447650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9718109600 msgr2=0x7f9718199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:39.120 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.117+0000 7f971d72e700 1 --2- 172.21.15.181:0/716447650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9718109600 0x7f9718199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:39.120 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.118+0000 7f971d72e700 1 -- 172.21.15.181:0/716447650 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971819a170 con 0x7f9718103eb0 2024-05-03T20:33:39.120 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.118+0000 7f971d72e700 1 --2- 172.21.15.181:0/716447650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9718103eb0 0x7f971819fd40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f970c000c00 tx=0x7f970c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:39.120 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.118+0000 7f9716ffd700 1 -- 172.21.15.181:0/716447650 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c027070 con 0x7f9718103eb0 2024-05-03T20:33:39.120 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.118+0000 7f9716ffd700 1 -- 172.21.15.181:0/716447650 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f970c022470 con 0x7f9718103eb0 2024-05-03T20:33:39.120 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.118+0000 7f9716ffd700 1 -- 172.21.15.181:0/716447650 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c017650 con 0x7f9718103eb0 2024-05-03T20:33:39.120 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.118+0000 7f9720193700 1 -- 172.21.15.181:0/716447650 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f971819a400 con 0x7f9718103eb0 2024-05-03T20:33:39.120 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.118+0000 7f9720193700 1 -- 172.21.15.181:0/716447650 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97181a66d0 con 0x7f9718103eb0 2024-05-03T20:33:39.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.119+0000 7f9720193700 1 -- 172.21.15.181:0/716447650 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9718104c10 con 0x7f9718103eb0 2024-05-03T20:33:39.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.120+0000 7f9716ffd700 1 -- 172.21.15.181:0/716447650 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f970c02a070 con 0x7f9718103eb0 2024-05-03T20:33:39.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.121+0000 7f9716ffd700 1 --2- 172.21.15.181:0/716447650 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f97000b2880 0x7f97000b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:39.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.121+0000 7f971df2f700 1 --2- 172.21.15.181:0/716447650 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f97000b2880 0x7f97000b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:39.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.121+0000 7f9716ffd700 1 -- 172.21.15.181:0/716447650 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7497+0+0 (secure 0 0 0) 0x7f970c0e31a0 con 0x7f9718103eb0 2024-05-03T20:33:39.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.121+0000 7f971df2f700 1 --2- 172.21.15.181:0/716447650 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f97000b2880 0x7f97000b4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9718101bc0 tx=0x7f9708009180 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:39.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.125+0000 7f9716ffd700 1 -- 172.21.15.181:0/716447650 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f970c013070 con 0x7f9718103eb0 2024-05-03T20:33:39.332 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.332+0000 7f9720193700 1 -- 172.21.15.181:0/716447650 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f971819aef0 con 0x7f9718103eb0 2024-05-03T20:33:39.679 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.679+0000 7f9716ffd700 1 -- 172.21.15.181:0/716447650 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v78) ==== 170+0+0 (secure 0 0 0) 0x7f970c0ab280 con 0x7f9718103eb0 2024-05-03T20:33:39.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.682+0000 7f9720193700 1 -- 172.21.15.181:0/716447650 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f97000b2880 msgr2=0x7f97000b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:39.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.682+0000 7f9720193700 1 --2- 172.21.15.181:0/716447650 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f97000b2880 0x7f97000b4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9718101bc0 tx=0x7f9708009180 comp rx=0 tx=0).stop 2024-05-03T20:33:39.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.682+0000 7f9720193700 1 -- 172.21.15.181:0/716447650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9718103eb0 msgr2=0x7f971819fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:39.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.682+0000 7f9720193700 1 --2- 172.21.15.181:0/716447650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9718103eb0 0x7f971819fd40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f970c000c00 tx=0x7f970c009580 comp rx=0 tx=0).stop 2024-05-03T20:33:39.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.682+0000 7f9720193700 1 -- 172.21.15.181:0/716447650 shutdown_connections 2024-05-03T20:33:39.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.682+0000 7f9720193700 1 --2- 172.21.15.181:0/716447650 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f97000b2880 0x7f97000b4d40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:39.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.682+0000 7f9720193700 1 --2- 172.21.15.181:0/716447650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9718109600 0x7f9718199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:39.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.682+0000 7f9720193700 1 --2- 172.21.15.181:0/716447650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97181034f0 0x7f971819f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:39.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.682+0000 7f9720193700 1 --2- 172.21.15.181:0/716447650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9718103eb0 0x7f971819fd40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:39.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.682+0000 7f9720193700 1 -- 172.21.15.181:0/716447650 >> 172.21.15.181:0/716447650 conn(0x7f971807a650 msgr2=0x7f9718102140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:39.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.682+0000 7f9720193700 1 -- 172.21.15.181:0/716447650 shutdown_connections 2024-05-03T20:33:39.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:39.682+0000 7f9720193700 1 -- 172.21.15.181:0/716447650 wait complete. 2024-05-03T20:33:39.683 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' created 2024-05-03T20:33:39.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:39 smithi181 ceph-mon[29801]: pgmap v278: 33 pgs: 1 creating+activating, 11 creating+peering, 1 active+clean, 20 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:39.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:39 smithi181 ceph-mon[29801]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T20:33:39.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:39 smithi181 ceph-mon[29801]: osdmap e77: 12 total, 12 up, 12 in 2024-05-03T20:33:39.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:39 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/716447650' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-03T20:33:40.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:39 smithi196 ceph-mon[32858]: pgmap v278: 33 pgs: 1 creating+activating, 11 creating+peering, 1 active+clean, 20 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:40.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:39 smithi196 ceph-mon[32858]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T20:33:40.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:39 smithi196 ceph-mon[32858]: osdmap e77: 12 total, 12 up, 12 in 2024-05-03T20:33:40.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:39 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/716447650' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-03T20:33:40.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:39 smithi204 ceph-mon[33595]: pgmap v278: 33 pgs: 1 creating+activating, 11 creating+peering, 1 active+clean, 20 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:40.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:39 smithi204 ceph-mon[33595]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T20:33:40.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:39 smithi204 ceph-mon[33595]: osdmap e77: 12 total, 12 up, 12 in 2024-05-03T20:33:40.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:39 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/716447650' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-03T20:33:40.332 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-03T20:33:40.577 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:33:40.949 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:40 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/716447650' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-03T20:33:40.949 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:40 smithi181 ceph-mon[29801]: osdmap e78: 12 total, 12 up, 12 in 2024-05-03T20:33:41.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:40 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/716447650' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-03T20:33:41.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:40 smithi196 ceph-mon[32858]: osdmap e78: 12 total, 12 up, 12 in 2024-05-03T20:33:41.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:40 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/716447650' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-03T20:33:41.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:40 smithi204 ceph-mon[33595]: osdmap e78: 12 total, 12 up, 12 in 2024-05-03T20:33:41.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:41 smithi181 ceph-mon[29801]: pgmap v281: 97 pgs: 1 creating+activating, 11 creating+peering, 6 active+clean, 79 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:41.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:41 smithi181 ceph-mon[29801]: osdmap e79: 12 total, 12 up, 12 in 2024-05-03T20:33:42.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:41 smithi196 ceph-mon[32858]: pgmap v281: 97 pgs: 1 creating+activating, 11 creating+peering, 6 active+clean, 79 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:42.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:41 smithi196 ceph-mon[32858]: osdmap e79: 12 total, 12 up, 12 in 2024-05-03T20:33:42.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:41 smithi204 ceph-mon[33595]: pgmap v281: 97 pgs: 1 creating+activating, 11 creating+peering, 6 active+clean, 79 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:42.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:41 smithi204 ceph-mon[33595]: osdmap e79: 12 total, 12 up, 12 in 2024-05-03T20:33:42.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.225+0000 7f577654d700 1 -- 172.21.15.181:0/1498068345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5770103f20 msgr2=0x7f5770110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:42.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.225+0000 7f577654d700 1 --2- 172.21.15.181:0/1498068345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5770103f20 0x7f5770110310 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f576800b0d0 tx=0x7f576800b3e0 comp rx=0 tx=0).stop 2024-05-03T20:33:42.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.225+0000 7f577654d700 1 -- 172.21.15.181:0/1498068345 shutdown_connections 2024-05-03T20:33:42.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.225+0000 7f577654d700 1 --2- 172.21.15.181:0/1498068345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5770103f20 0x7f5770110310 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:42.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.225+0000 7f577654d700 1 --2- 172.21.15.181:0/1498068345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5770109540 0x7f5770109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:42.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.225+0000 7f577654d700 1 --2- 172.21.15.181:0/1498068345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5770103560 0x7f57701039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:42.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.225+0000 7f577654d700 1 -- 172.21.15.181:0/1498068345 >> 172.21.15.181:0/1498068345 conn(0x7f577007a660 msgr2=0x7f577007aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:42.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.225+0000 7f577654d700 1 -- 172.21.15.181:0/1498068345 shutdown_connections 2024-05-03T20:33:42.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.225+0000 7f577654d700 1 -- 172.21.15.181:0/1498068345 wait complete. 2024-05-03T20:33:42.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.226+0000 7f577654d700 1 Processor -- start 2024-05-03T20:33:42.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.226+0000 7f577654d700 1 -- start start 2024-05-03T20:33:42.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.226+0000 7f577654d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5770103560 0x7f577019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:42.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.226+0000 7f577654d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5770103f20 0x7f577019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:42.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.226+0000 7f577654d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5770109540 0x7f57701999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:42.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.226+0000 7f577654d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f57701136c0 con 0x7f5770109540 2024-05-03T20:33:42.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.226+0000 7f577654d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5770113540 con 0x7f5770103f20 2024-05-03T20:33:42.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.226+0000 7f577654d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5770113840 con 0x7f5770103560 2024-05-03T20:33:42.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.226+0000 7f5774aea700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5770109540 0x7f57701999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:42.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.226+0000 7f5774aea700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5770109540 0x7f57701999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35996/0 (socket says 172.21.15.181:35996) 2024-05-03T20:33:42.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.226+0000 7f5774aea700 1 -- 172.21.15.181:0/2110320485 learned_addr learned my addr 172.21.15.181:0/2110320485 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:33:42.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.227+0000 7f576ffff700 1 --2- 172.21.15.181:0/2110320485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5770103560 0x7f577019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:42.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.227+0000 7f576f7fe700 1 --2- 172.21.15.181:0/2110320485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5770103f20 0x7f577019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:42.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.227+0000 7f5774aea700 1 -- 172.21.15.181:0/2110320485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5770103560 msgr2=0x7f577019f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:42.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.227+0000 7f5774aea700 1 --2- 172.21.15.181:0/2110320485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5770103560 0x7f577019f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:42.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.227+0000 7f5774aea700 1 -- 172.21.15.181:0/2110320485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5770103f20 msgr2=0x7f577019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:42.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.227+0000 7f5774aea700 1 --2- 172.21.15.181:0/2110320485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5770103f20 0x7f577019fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:42.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.227+0000 7f5774aea700 1 -- 172.21.15.181:0/2110320485 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f577019a100 con 0x7f5770109540 2024-05-03T20:33:42.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.227+0000 7f576ffff700 1 --2- 172.21.15.181:0/2110320485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5770103560 0x7f577019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:33:42.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.227+0000 7f5774aea700 1 --2- 172.21.15.181:0/2110320485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5770109540 0x7f57701999a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f57680095c0 tx=0x7f5768007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:42.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.227+0000 7f576d7fa700 1 -- 172.21.15.181:0/2110320485 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576801f070 con 0x7f5770109540 2024-05-03T20:33:42.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.227+0000 7f576d7fa700 1 -- 172.21.15.181:0/2110320485 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5768003c00 con 0x7f5770109540 2024-05-03T20:33:42.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.227+0000 7f577654d700 1 -- 172.21.15.181:0/2110320485 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f577019a390 con 0x7f5770109540 2024-05-03T20:33:42.258 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.227+0000 7f576d7fa700 1 -- 172.21.15.181:0/2110320485 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576801b6a0 con 0x7f5770109540 2024-05-03T20:33:42.258 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.227+0000 7f577654d700 1 -- 172.21.15.181:0/2110320485 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57701a66b0 con 0x7f5770109540 2024-05-03T20:33:42.258 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.228+0000 7f577654d700 1 -- 172.21.15.181:0/2110320485 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f577010b9d0 con 0x7f5770109540 2024-05-03T20:33:42.258 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.229+0000 7f576d7fa700 1 -- 172.21.15.181:0/2110320485 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f5768022070 con 0x7f5770109540 2024-05-03T20:33:42.258 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.230+0000 7f576d7fa700 1 --2- 172.21.15.181:0/2110320485 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f57580b2880 0x7f57580b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:42.258 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.230+0000 7f576ffff700 1 --2- 172.21.15.181:0/2110320485 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f57580b2880 0x7f57580b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:42.258 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.230+0000 7f576d7fa700 1 -- 172.21.15.181:0/2110320485 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7879+0+0 (secure 0 0 0) 0x7f576800f020 con 0x7f5770109540 2024-05-03T20:33:42.258 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.231+0000 7f576ffff700 1 --2- 172.21.15.181:0/2110320485 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f57580b2880 0x7f57580b4d40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5760005fd0 tx=0x7f5760005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:42.258 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.234+0000 7f576d7fa700 1 -- 172.21.15.181:0/2110320485 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5768029020 con 0x7f5770109540 2024-05-03T20:33:42.485 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.486+0000 7f577654d700 1 -- 172.21.15.181:0/2110320485 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5770104bb0 con 0x7f5770109540 2024-05-03T20:33:42.701 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.701+0000 7f576d7fa700 1 -- 172.21.15.181:0/2110320485 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f57680ab710 con 0x7f5770109540 2024-05-03T20:33:42.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.704+0000 7f577654d700 1 -- 172.21.15.181:0/2110320485 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f57580b2880 msgr2=0x7f57580b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:42.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.704+0000 7f577654d700 1 --2- 172.21.15.181:0/2110320485 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f57580b2880 0x7f57580b4d40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5760005fd0 tx=0x7f5760005f00 comp rx=0 tx=0).stop 2024-05-03T20:33:42.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.704+0000 7f577654d700 1 -- 172.21.15.181:0/2110320485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5770109540 msgr2=0x7f57701999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:42.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.704+0000 7f577654d700 1 --2- 172.21.15.181:0/2110320485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5770109540 0x7f57701999a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f57680095c0 tx=0x7f5768007ad0 comp rx=0 tx=0).stop 2024-05-03T20:33:42.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.704+0000 7f577654d700 1 -- 172.21.15.181:0/2110320485 shutdown_connections 2024-05-03T20:33:42.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.704+0000 7f577654d700 1 --2- 172.21.15.181:0/2110320485 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f57580b2880 0x7f57580b4d40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:42.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.704+0000 7f577654d700 1 --2- 172.21.15.181:0/2110320485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5770109540 0x7f57701999a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:42.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.704+0000 7f577654d700 1 --2- 172.21.15.181:0/2110320485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5770103560 0x7f577019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:42.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.704+0000 7f577654d700 1 --2- 172.21.15.181:0/2110320485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5770103f20 0x7f577019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:42.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.704+0000 7f577654d700 1 -- 172.21.15.181:0/2110320485 >> 172.21.15.181:0/2110320485 conn(0x7f577007a660 msgr2=0x7f5770102260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:42.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.704+0000 7f577654d700 1 -- 172.21.15.181:0/2110320485 shutdown_connections 2024-05-03T20:33:42.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:42.704+0000 7f577654d700 1 -- 172.21.15.181:0/2110320485 wait complete. 2024-05-03T20:33:42.705 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 2 and data pool 3 2024-05-03T20:33:42.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:42 smithi181 ceph-mon[29801]: osdmap e80: 12 total, 12 up, 12 in 2024-05-03T20:33:42.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:42 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/2110320485' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-03T20:33:42.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:42 smithi181 ceph-mon[29801]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-03T20:33:43.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:42 smithi204 ceph-mon[33595]: osdmap e80: 12 total, 12 up, 12 in 2024-05-03T20:33:43.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:42 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/2110320485' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-03T20:33:43.015 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:42 smithi204 ceph-mon[33595]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-03T20:33:43.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:42 smithi196 ceph-mon[32858]: osdmap e80: 12 total, 12 up, 12 in 2024-05-03T20:33:43.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:42 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/2110320485' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-03T20:33:43.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:42 smithi196 ceph-mon[32858]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-03T20:33:43.272 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T20:33:43.510 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:33:43.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:43 smithi181 ceph-mon[29801]: pgmap v284: 97 pgs: 1 creating+activating, 20 creating+peering, 36 active+clean, 40 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:43.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:43 smithi181 ceph-mon[29801]: osdmap e81: 12 total, 12 up, 12 in 2024-05-03T20:33:43.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:43 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/2110320485' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-03T20:33:43.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:43 smithi181 ceph-mon[29801]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-03T20:33:43.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:43 smithi181 ceph-mon[29801]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-03T20:33:44.106 INFO:journalctl@ceph.mds.b.smithi196.stdout:May 03 20:33:43 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-b[63557]: 2024-05-03T20:33:43.915+0000 7f1d4e06d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-03T20:33:44.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:43 smithi196 ceph-mon[32858]: pgmap v284: 97 pgs: 1 creating+activating, 20 creating+peering, 36 active+clean, 40 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:44.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:43 smithi196 ceph-mon[32858]: osdmap e81: 12 total, 12 up, 12 in 2024-05-03T20:33:44.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:43 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/2110320485' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-03T20:33:44.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:43 smithi196 ceph-mon[32858]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-03T20:33:44.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:43 smithi196 ceph-mon[32858]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-03T20:33:44.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:43 smithi204 ceph-mon[33595]: pgmap v284: 97 pgs: 1 creating+activating, 20 creating+peering, 36 active+clean, 40 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:33:44.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:43 smithi204 ceph-mon[33595]: osdmap e81: 12 total, 12 up, 12 in 2024-05-03T20:33:44.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:43 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/2110320485' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-03T20:33:44.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:43 smithi204 ceph-mon[33595]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-03T20:33:44.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:43 smithi204 ceph-mon[33595]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-03T20:33:44.864 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:44 smithi181 ceph-mon[29801]: mds.? [v2:172.21.15.196:6838/1182826743,v1:172.21.15.196:6839/1182826743] up:active 2024-05-03T20:33:44.864 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:44 smithi181 ceph-mon[29801]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-03T20:33:44.864 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:44 smithi181 ceph-mon[29801]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-03T20:33:44.864 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:44 smithi181 ceph-mon[29801]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T20:33:45.007 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:44 smithi196 ceph-mon[32858]: mds.? [v2:172.21.15.196:6838/1182826743,v1:172.21.15.196:6839/1182826743] up:active 2024-05-03T20:33:45.007 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:44 smithi196 ceph-mon[32858]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-03T20:33:45.007 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:44 smithi196 ceph-mon[32858]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-03T20:33:45.007 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:44 smithi196 ceph-mon[32858]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T20:33:45.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.103+0000 7f079eea6700 1 -- 172.21.15.181:0/1879530929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0798102390 msgr2=0x7f0798102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:45.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.103+0000 7f079eea6700 1 --2- 172.21.15.181:0/1879530929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0798102390 0x7f0798102810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0780009990 tx=0x7f0780009ca0 comp rx=0 tx=0).stop 2024-05-03T20:33:45.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.103+0000 7f079eea6700 1 -- 172.21.15.181:0/1879530929 shutdown_connections 2024-05-03T20:33:45.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.103+0000 7f079eea6700 1 --2- 172.21.15.181:0/1879530929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0798102d50 0x7f079810f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:45.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.103+0000 7f079eea6700 1 --2- 172.21.15.181:0/1879530929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0798108400 0x7f07981087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:45.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.103+0000 7f079eea6700 1 --2- 172.21.15.181:0/1879530929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0798102390 0x7f0798102810 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:45.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.103+0000 7f079eea6700 1 -- 172.21.15.181:0/1879530929 >> 172.21.15.181:0/1879530929 conn(0x7f0798079310 msgr2=0x7f0798079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:45.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.103+0000 7f079eea6700 1 -- 172.21.15.181:0/1879530929 shutdown_connections 2024-05-03T20:33:45.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.103+0000 7f079eea6700 1 -- 172.21.15.181:0/1879530929 wait complete. 2024-05-03T20:33:45.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.104+0000 7f079eea6700 1 Processor -- start 2024-05-03T20:33:45.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.104+0000 7f079eea6700 1 -- start start 2024-05-03T20:33:45.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.104+0000 7f079eea6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0798102390 0x7f079819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:45.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.104+0000 7f079eea6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0798102d50 0x7f079819afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:45.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.104+0000 7f079eea6700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0798108400 0x7f07981a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:45.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.104+0000 7f079eea6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f07981125d0 con 0x7f0798102390 2024-05-03T20:33:45.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.104+0000 7f079eea6700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0798112450 con 0x7f0798108400 2024-05-03T20:33:45.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.104+0000 7f079eea6700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0798112750 con 0x7f0798102d50 2024-05-03T20:33:45.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.104+0000 7f079cc42700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0798102390 0x7f079819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:45.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.104+0000 7f079cc42700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0798102390 0x7f079819aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36018/0 (socket says 172.21.15.181:36018) 2024-05-03T20:33:45.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.104+0000 7f079cc42700 1 -- 172.21.15.181:0/4065912112 learned_addr learned my addr 172.21.15.181:0/4065912112 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:33:45.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.104+0000 7f0797fff700 1 --2- 172.21.15.181:0/4065912112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0798102d50 0x7f079819afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:45.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.104+0000 7f079cc42700 1 -- 172.21.15.181:0/4065912112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0798102d50 msgr2=0x7f079819afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:45.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.104+0000 7f079cc42700 1 --2- 172.21.15.181:0/4065912112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0798102d50 0x7f079819afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:45.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.104+0000 7f079cc42700 1 -- 172.21.15.181:0/4065912112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0798108400 msgr2=0x7f07981a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:45.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.104+0000 7f079cc42700 1 --2- 172.21.15.181:0/4065912112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0798108400 0x7f07981a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:45.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.104+0000 7f079cc42700 1 -- 172.21.15.181:0/4065912112 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07981a6030 con 0x7f0798102390 2024-05-03T20:33:45.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.104+0000 7f0797fff700 1 --2- 172.21.15.181:0/4065912112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0798102d50 0x7f079819afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:33:45.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.104+0000 7f079cc42700 1 --2- 172.21.15.181:0/4065912112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0798102390 0x7f079819aa70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f078800b530 tx=0x7f078800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:45.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.104+0000 7f0795ffb700 1 -- 172.21.15.181:0/4065912112 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0788015020 con 0x7f0798102390 2024-05-03T20:33:45.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.105+0000 7f0795ffb700 1 -- 172.21.15.181:0/4065912112 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0788011ce0 con 0x7f0798102390 2024-05-03T20:33:45.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.105+0000 7f079eea6700 1 -- 172.21.15.181:0/4065912112 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07981a6260 con 0x7f0798102390 2024-05-03T20:33:45.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.105+0000 7f0795ffb700 1 -- 172.21.15.181:0/4065912112 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0788010d20 con 0x7f0798102390 2024-05-03T20:33:45.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.105+0000 7f079eea6700 1 -- 172.21.15.181:0/4065912112 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07981a6770 con 0x7f0798102390 2024-05-03T20:33:45.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.106+0000 7f079eea6700 1 -- 172.21.15.181:0/4065912112 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f079810a970 con 0x7f0798102390 2024-05-03T20:33:45.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.108+0000 7f0795ffb700 1 -- 172.21.15.181:0/4065912112 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f0788018020 con 0x7f0798102390 2024-05-03T20:33:45.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.108+0000 7f0795ffb700 1 --2- 172.21.15.181:0/4065912112 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f07840b2880 0x7f07840b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:45.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.108+0000 7f0797fff700 1 --2- 172.21.15.181:0/4065912112 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f07840b2880 0x7f07840b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:45.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.109+0000 7f0795ffb700 1 -- 172.21.15.181:0/4065912112 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f07880e1e60 con 0x7f0798102390 2024-05-03T20:33:45.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.109+0000 7f0797fff700 1 --2- 172.21.15.181:0/4065912112 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f07840b2880 0x7f07840b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0780006010 tx=0x7f078000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:45.111 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.112+0000 7f0795ffb700 1 -- 172.21.15.181:0/4065912112 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f078801f020 con 0x7f0798102390 2024-05-03T20:33:45.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:44 smithi204 ceph-mon[33595]: mds.? [v2:172.21.15.196:6838/1182826743,v1:172.21.15.196:6839/1182826743] up:active 2024-05-03T20:33:45.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:44 smithi204 ceph-mon[33595]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-03T20:33:45.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:44 smithi204 ceph-mon[33595]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-03T20:33:45.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:44 smithi204 ceph-mon[33595]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T20:33:45.314 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.314+0000 7f079eea6700 1 -- 172.21.15.181:0/4065912112 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f079806a190 con 0x7f0798102390 2024-05-03T20:33:45.315 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.316+0000 7f0795ffb700 1 -- 172.21.15.181:0/4065912112 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22240 (secure 0 0 0) 0x7f07880a9cd0 con 0x7f0798102390 2024-05-03T20:33:45.316 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:33:45.316 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":81,"fsid":"1cd058ee-098b-11ef-bc97-c7b262605968","created":"2024-05-03T20:25:39.427815+0000","modified":"2024-05-03T20:33:42.488002+0000","last_up_change":"2024-05-03T20:32:38.854730+0000","last_in_change":"2024-05-03T20:32:18.787388+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T20:29:01.855187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T20:33:35.737273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T20:33:39.337556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5f594d97-10a4-4f77-aebc-c00dbe92a4af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6803","nonce":1683973278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6805","nonce":1683973278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6809","nonce":1683973278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6807","nonce":1683973278}]},"public_addr":"172.21.15.181:6803/1683973278","cluster_addr":"172.21.15.181:6805/1683973278","heartbeat_back_addr":"172.21.15.181:6809/1683973278","heartbeat_front_addr":"172.21.15.181:6807/1683973278","state":["exists","up"]},{"osd":1,"uuid":"9726d641-8930-4b45-9192-6ee2b6bfcdcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6801","nonce":2136873375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6803","nonce":2136873375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6807","nonce":2136873375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6805","nonce":2136873375}]},"public_addr":"172.21.15.196:6801/2136873375","cluster_addr":"172.21.15.196:6803/2136873375","heartbeat_back_addr":"172.21.15.196:6807/2136873375","heartbeat_front_addr":"172.21.15.196:6805/2136873375","state":["exists","up"]},{"osd":2,"uuid":"3d5573cc-37c1-4c3b-a61e-2363ef3ff964","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6801","nonce":3291673137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6803","nonce":3291673137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6807","nonce":3291673137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6805","nonce":3291673137}]},"public_addr":"172.21.15.204:6801/3291673137","cluster_addr":"172.21.15.204:6803/3291673137","heartbeat_back_addr":"172.21.15.204:6807/3291673137","heartbeat_front_addr":"172.21.15.204:6805/3291673137","state":["exists","up"]},{"osd":3,"uuid":"b9b205e3-a335-4771-9a66-55cfe6a8c31a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6811","nonce":29064971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6813","nonce":29064971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6817","nonce":29064971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6815","nonce":29064971}]},"public_addr":"172.21.15.181:6811/29064971","cluster_addr":"172.21.15.181:6813/29064971","heartbeat_back_addr":"172.21.15.181:6817/29064971","heartbeat_front_addr":"172.21.15.181:6815/29064971","state":["exists","up"]},{"osd":4,"uuid":"550f01ef-1947-4f9f-b759-34e3ea831839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6809","nonce":1145408995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6811","nonce":1145408995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6815","nonce":1145408995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6813","nonce":1145408995}]},"public_addr":"172.21.15.196:6809/1145408995","cluster_addr":"172.21.15.196:6811/1145408995","heartbeat_back_addr":"172.21.15.196:6815/1145408995","heartbeat_front_addr":"172.21.15.196:6813/1145408995","state":["exists","up"]},{"osd":5,"uuid":"6b74cb3a-f39a-4131-8793-1d5286856aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6809","nonce":855680559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6811","nonce":855680559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6815","nonce":855680559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6813","nonce":855680559}]},"public_addr":"172.21.15.204:6809/855680559","cluster_addr":"172.21.15.204:6811/855680559","heartbeat_back_addr":"172.21.15.204:6815/855680559","heartbeat_front_addr":"172.21.15.204:6813/855680559","state":["exists","up"]},{"osd":6,"uuid":"dbe19bdb-08eb-40aa-842b-0710963d4497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6819","nonce":1561969338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6821","nonce":1561969338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6825","nonce":1561969338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6823","nonce":1561969338}]},"public_addr":"172.21.15.181:6819/1561969338","cluster_addr":"172.21.15.181:6821/1561969338","heartbeat_back_addr":"172.21.15.181:6825/1561969338","heartbeat_front_addr":"172.21.15.181:6823/1561969338","state":["exists","up"]},{"osd":7,"uuid":"016c145c-ad67-42ac-9775-ca2840790abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6817","nonce":824890163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6819","nonce":824890163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6823","nonce":824890163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6821","nonce":824890163}]},"public_addr":"172.21.15.196:6817/824890163","cluster_addr":"172.21.15.196:6819/824890163","heartbeat_back_addr":"172.21.15.196:6823/824890163","heartbeat_front_addr":"172.21.15.196:6821/824890163","state":["exists","up"]},{"osd":8,"uuid":"10235638-c1f1-41d3-8e24-f8b2344bfeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6817","nonce":3566909552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6819","nonce":3566909552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6823","nonce":3566909552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6821","nonce":3566909552}]},"public_addr":"172.21.15.204:6817/3566909552","cluster_addr":"172.21.15.204:6819/3566909552","heartbeat_back_addr":"172.21.15.204:6823/3566909552","heartbeat_front_addr":"172.21.15.204:6821/3566909552","state":["exists","up"]},{"osd":9,"uuid":"96ce5aa6-e6a9-4d83-8562-589ac94c8cd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6827","nonce":241693472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6829","nonce":241693472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6833","nonce":241693472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6831","nonce":241693472}]},"public_addr":"172.21.15.181:6827/241693472","cluster_addr":"172.21.15.181:6829/241693472","heartbeat_back_addr":"172.21.15.181:6833/241693472","heartbeat_front_addr":"172.21.15.181:6831/241693472","state":["exists","up"]},{"osd":10,"uuid":"d6c9ffab-731c-46d5-88cd-5f21ba5acfa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6825","nonce":3980645995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6827","nonce":3980645995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6831","nonce":3980645995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6829","nonce":3980645995}]},"public_addr":"172.21.15.196:6825/3980645995","cluster_addr":"172.21.15.196:6827/3980645995","heartbeat_back_addr":"172.21.15.196:6831/3980645995","heartbeat_front_addr":"172.21.15.196:6829/3980645995","state":["exists","up"]},{"osd":11,"uuid":"3f58a2d6-bbbb-4e4d-bf83-2c886dd9484b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6825","nonce":488710574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6827","nonce":488710574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6831","nonce":488710574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6829","nonce":488710574}]},"public_addr":"172.21.15.204:6825/488710574","cluster_addr":"172.21.15.204:6827/488710574","heartbeat_back_addr":"172.21.15.204:6831/488710574","heartbeat_front_addr":"172.21.15.204:6829/488710574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:28:09.700091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:28:33.838568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:28:57.265536+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:29:21.720066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:29:45.480447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:30:09.558384+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:30:34.253331+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:30:57.986111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:31:22.470746+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:31:47.548762+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:32:11.285657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:32:36.888004+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6800/2389231557":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/3398276284":"2024-05-04T20:26:06.561713+0000","172.21.15.181:6801/2389231557":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/2327733609":"2024-05-04T20:26:06.561713+0000","172.21.15.181:0/628540759":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/3387163329":"2024-05-04T20:26:06.561713+0000","172.21.15.181:6800/1802984663":"2024-05-04T20:26:06.561713+0000","172.21.15.181:0/67924109":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/3693389657":"2024-05-04T20:26:33.766772+0000","172.21.15.181:6801/1802984663":"2024-05-04T20:26:06.561713+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T20:33:45.318 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.318+0000 7f079eea6700 1 -- 172.21.15.181:0/4065912112 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f07840b2880 msgr2=0x7f07840b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:45.318 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.318+0000 7f079eea6700 1 --2- 172.21.15.181:0/4065912112 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f07840b2880 0x7f07840b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0780006010 tx=0x7f078000b540 comp rx=0 tx=0).stop 2024-05-03T20:33:45.318 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.318+0000 7f079eea6700 1 -- 172.21.15.181:0/4065912112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0798102390 msgr2=0x7f079819aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:45.318 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.318+0000 7f079eea6700 1 --2- 172.21.15.181:0/4065912112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0798102390 0x7f079819aa70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f078800b530 tx=0x7f078800b8f0 comp rx=0 tx=0).stop 2024-05-03T20:33:45.318 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.318+0000 7f079eea6700 1 -- 172.21.15.181:0/4065912112 shutdown_connections 2024-05-03T20:33:45.318 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.318+0000 7f079eea6700 1 --2- 172.21.15.181:0/4065912112 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f07840b2880 0x7f07840b4d40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:45.318 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.318+0000 7f079eea6700 1 --2- 172.21.15.181:0/4065912112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0798108400 0x7f07981a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:45.318 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.318+0000 7f079eea6700 1 --2- 172.21.15.181:0/4065912112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0798102390 0x7f079819aa70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:45.318 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.318+0000 7f079eea6700 1 --2- 172.21.15.181:0/4065912112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0798102d50 0x7f079819afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:45.319 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.318+0000 7f079eea6700 1 -- 172.21.15.181:0/4065912112 >> 172.21.15.181:0/4065912112 conn(0x7f0798079310 msgr2=0x7f0798100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:45.319 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.318+0000 7f079eea6700 1 -- 172.21.15.181:0/4065912112 shutdown_connections 2024-05-03T20:33:45.319 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:45.318+0000 7f079eea6700 1 -- 172.21.15.181:0/4065912112 wait complete. 2024-05-03T20:33:45.932 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:45 smithi181 ceph-mon[29801]: pgmap v286: 97 pgs: 12 creating+peering, 70 active+clean, 15 unknown; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 737 B/s wr, 1 op/s 2024-05-03T20:33:45.932 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:45 smithi181 ceph-mon[29801]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-03T20:33:45.932 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:45 smithi181 ceph-mon[29801]: Cluster is now healthy 2024-05-03T20:33:45.932 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:45 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/4065912112' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T20:33:45.959 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T20:33:46.023 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:45 smithi196 ceph-mon[32858]: pgmap v286: 97 pgs: 12 creating+peering, 70 active+clean, 15 unknown; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 737 B/s wr, 1 op/s 2024-05-03T20:33:46.023 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:45 smithi196 ceph-mon[32858]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-03T20:33:46.023 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:45 smithi196 ceph-mon[32858]: Cluster is now healthy 2024-05-03T20:33:46.023 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:45 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/4065912112' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T20:33:46.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:45 smithi204 ceph-mon[33595]: pgmap v286: 97 pgs: 12 creating+peering, 70 active+clean, 15 unknown; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 737 B/s wr, 1 op/s 2024-05-03T20:33:46.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:45 smithi204 ceph-mon[33595]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-03T20:33:46.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:45 smithi204 ceph-mon[33595]: Cluster is now healthy 2024-05-03T20:33:46.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:45 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/4065912112' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T20:33:46.197 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:33:47.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.794+0000 7fcc50dbf700 1 -- 172.21.15.181:0/3907635459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc4c06c820 msgr2=0x7fcc4c1080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:47.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.794+0000 7fcc50dbf700 1 --2- 172.21.15.181:0/3907635459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc4c06c820 0x7fcc4c1080e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcc400099c0 tx=0x7fcc40009cd0 comp rx=0 tx=0).stop 2024-05-03T20:33:47.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.795+0000 7fcc50dbf700 1 -- 172.21.15.181:0/3907635459 shutdown_connections 2024-05-03T20:33:47.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.795+0000 7fcc50dbf700 1 --2- 172.21.15.181:0/3907635459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc4c1087c0 0x7fcc4c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:47.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.795+0000 7fcc50dbf700 1 --2- 172.21.15.181:0/3907635459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc4c06be70 0x7fcc4c06c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:47.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.795+0000 7fcc50dbf700 1 --2- 172.21.15.181:0/3907635459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc4c06c820 0x7fcc4c1080e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:47.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.795+0000 7fcc50dbf700 1 -- 172.21.15.181:0/3907635459 >> 172.21.15.181:0/3907635459 conn(0x7fcc4c0783f0 msgr2=0x7fcc4c078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:47.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.795+0000 7fcc50dbf700 1 -- 172.21.15.181:0/3907635459 shutdown_connections 2024-05-03T20:33:47.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.795+0000 7fcc50dbf700 1 -- 172.21.15.181:0/3907635459 wait complete. 2024-05-03T20:33:47.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.795+0000 7fcc50dbf700 1 Processor -- start 2024-05-03T20:33:47.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.795+0000 7fcc50dbf700 1 -- start start 2024-05-03T20:33:47.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.796+0000 7fcc50dbf700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc4c06be70 0x7fcc4c19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:47.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.796+0000 7fcc50dbf700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc4c06c820 0x7fcc4c19fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:47.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.796+0000 7fcc50dbf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc4c1087c0 0x7fcc4c1999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:47.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.796+0000 7fcc50dbf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c113690 con 0x7fcc4c1087c0 2024-05-03T20:33:47.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.796+0000 7fcc50dbf700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c113510 con 0x7fcc4c06c820 2024-05-03T20:33:47.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.796+0000 7fcc50dbf700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c113810 con 0x7fcc4c06be70 2024-05-03T20:33:47.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.796+0000 7fcc4ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc4c1087c0 0x7fcc4c1999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:47.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.796+0000 7fcc4ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc4c1087c0 0x7fcc4c1999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39368/0 (socket says 172.21.15.181:39368) 2024-05-03T20:33:47.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.796+0000 7fcc4ad9d700 1 -- 172.21.15.181:0/3761565833 learned_addr learned my addr 172.21.15.181:0/3761565833 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:33:47.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.796+0000 7fcc49d9b700 1 --2- 172.21.15.181:0/3761565833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc4c06c820 0x7fcc4c19fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:47.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.796+0000 7fcc4a59c700 1 --2- 172.21.15.181:0/3761565833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc4c06be70 0x7fcc4c19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:47.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.796+0000 7fcc4ad9d700 1 -- 172.21.15.181:0/3761565833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc4c06be70 msgr2=0x7fcc4c19f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:47.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.796+0000 7fcc4ad9d700 1 --2- 172.21.15.181:0/3761565833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc4c06be70 0x7fcc4c19f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:47.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.796+0000 7fcc4ad9d700 1 -- 172.21.15.181:0/3761565833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc4c06c820 msgr2=0x7fcc4c19fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:47.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.796+0000 7fcc4ad9d700 1 --2- 172.21.15.181:0/3761565833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc4c06c820 0x7fcc4c19fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:47.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.796+0000 7fcc4ad9d700 1 -- 172.21.15.181:0/3761565833 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4c19a130 con 0x7fcc4c1087c0 2024-05-03T20:33:47.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.796+0000 7fcc4ad9d700 1 --2- 172.21.15.181:0/3761565833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc4c1087c0 0x7fcc4c1999d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcc44007ee0 tx=0x7fcc4400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:47.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.796+0000 7fcc3b7fe700 1 -- 172.21.15.181:0/3761565833 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44004bb0 con 0x7fcc4c1087c0 2024-05-03T20:33:47.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.797+0000 7fcc50dbf700 1 -- 172.21.15.181:0/3761565833 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc4c19a420 con 0x7fcc4c1087c0 2024-05-03T20:33:47.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.797+0000 7fcc3b7fe700 1 -- 172.21.15.181:0/3761565833 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc44004d50 con 0x7fcc4c1087c0 2024-05-03T20:33:47.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.797+0000 7fcc3b7fe700 1 -- 172.21.15.181:0/3761565833 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44005920 con 0x7fcc4c1087c0 2024-05-03T20:33:47.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.797+0000 7fcc50dbf700 1 -- 172.21.15.181:0/3761565833 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc4c1a66e0 con 0x7fcc4c1087c0 2024-05-03T20:33:47.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.798+0000 7fcc50dbf700 1 -- 172.21.15.181:0/3761565833 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc4c06a190 con 0x7fcc4c1087c0 2024-05-03T20:33:47.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.799+0000 7fcc3b7fe700 1 -- 172.21.15.181:0/3761565833 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fcc4401a020 con 0x7fcc4c1087c0 2024-05-03T20:33:47.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.799+0000 7fcc3b7fe700 1 --2- 172.21.15.181:0/3761565833 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fcc340b2880 0x7fcc340b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:47.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.799+0000 7fcc4a59c700 1 --2- 172.21.15.181:0/3761565833 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fcc340b2880 0x7fcc340b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:47.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.799+0000 7fcc3b7fe700 1 -- 172.21.15.181:0/3761565833 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fcc440e1d40 con 0x7fcc4c1087c0 2024-05-03T20:33:47.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.800+0000 7fcc4a59c700 1 --2- 172.21.15.181:0/3761565833 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fcc340b2880 0x7fcc340b4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c19ab10 tx=0x7fcc3c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:47.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:47.803+0000 7fcc3b7fe700 1 -- 172.21.15.181:0/3761565833 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fcc44010070 con 0x7fcc4c1087c0 2024-05-03T20:33:47.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:47 smithi181 ceph-mon[29801]: pgmap v287: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 5 op/s 2024-05-03T20:33:48.011 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:48.011+0000 7fcc50dbf700 1 -- 172.21.15.181:0/3761565833 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcc4c02e510 con 0x7fcc4c1087c0 2024-05-03T20:33:48.012 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:48.013+0000 7fcc3b7fe700 1 -- 172.21.15.181:0/3761565833 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22240 (secure 0 0 0) 0x7fcc440a9bb0 con 0x7fcc4c1087c0 2024-05-03T20:33:48.013 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:33:48.013 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":81,"fsid":"1cd058ee-098b-11ef-bc97-c7b262605968","created":"2024-05-03T20:25:39.427815+0000","modified":"2024-05-03T20:33:42.488002+0000","last_up_change":"2024-05-03T20:32:38.854730+0000","last_in_change":"2024-05-03T20:32:18.787388+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T20:29:01.855187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T20:33:35.737273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T20:33:39.337556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5f594d97-10a4-4f77-aebc-c00dbe92a4af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6803","nonce":1683973278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6805","nonce":1683973278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6809","nonce":1683973278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6807","nonce":1683973278}]},"public_addr":"172.21.15.181:6803/1683973278","cluster_addr":"172.21.15.181:6805/1683973278","heartbeat_back_addr":"172.21.15.181:6809/1683973278","heartbeat_front_addr":"172.21.15.181:6807/1683973278","state":["exists","up"]},{"osd":1,"uuid":"9726d641-8930-4b45-9192-6ee2b6bfcdcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6801","nonce":2136873375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6803","nonce":2136873375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6807","nonce":2136873375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6805","nonce":2136873375}]},"public_addr":"172.21.15.196:6801/2136873375","cluster_addr":"172.21.15.196:6803/2136873375","heartbeat_back_addr":"172.21.15.196:6807/2136873375","heartbeat_front_addr":"172.21.15.196:6805/2136873375","state":["exists","up"]},{"osd":2,"uuid":"3d5573cc-37c1-4c3b-a61e-2363ef3ff964","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6801","nonce":3291673137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6803","nonce":3291673137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6807","nonce":3291673137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6805","nonce":3291673137}]},"public_addr":"172.21.15.204:6801/3291673137","cluster_addr":"172.21.15.204:6803/3291673137","heartbeat_back_addr":"172.21.15.204:6807/3291673137","heartbeat_front_addr":"172.21.15.204:6805/3291673137","state":["exists","up"]},{"osd":3,"uuid":"b9b205e3-a335-4771-9a66-55cfe6a8c31a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6811","nonce":29064971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6813","nonce":29064971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6817","nonce":29064971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6815","nonce":29064971}]},"public_addr":"172.21.15.181:6811/29064971","cluster_addr":"172.21.15.181:6813/29064971","heartbeat_back_addr":"172.21.15.181:6817/29064971","heartbeat_front_addr":"172.21.15.181:6815/29064971","state":["exists","up"]},{"osd":4,"uuid":"550f01ef-1947-4f9f-b759-34e3ea831839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6809","nonce":1145408995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6811","nonce":1145408995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6815","nonce":1145408995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6813","nonce":1145408995}]},"public_addr":"172.21.15.196:6809/1145408995","cluster_addr":"172.21.15.196:6811/1145408995","heartbeat_back_addr":"172.21.15.196:6815/1145408995","heartbeat_front_addr":"172.21.15.196:6813/1145408995","state":["exists","up"]},{"osd":5,"uuid":"6b74cb3a-f39a-4131-8793-1d5286856aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6809","nonce":855680559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6811","nonce":855680559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6815","nonce":855680559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6813","nonce":855680559}]},"public_addr":"172.21.15.204:6809/855680559","cluster_addr":"172.21.15.204:6811/855680559","heartbeat_back_addr":"172.21.15.204:6815/855680559","heartbeat_front_addr":"172.21.15.204:6813/855680559","state":["exists","up"]},{"osd":6,"uuid":"dbe19bdb-08eb-40aa-842b-0710963d4497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6819","nonce":1561969338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6821","nonce":1561969338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6825","nonce":1561969338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6823","nonce":1561969338}]},"public_addr":"172.21.15.181:6819/1561969338","cluster_addr":"172.21.15.181:6821/1561969338","heartbeat_back_addr":"172.21.15.181:6825/1561969338","heartbeat_front_addr":"172.21.15.181:6823/1561969338","state":["exists","up"]},{"osd":7,"uuid":"016c145c-ad67-42ac-9775-ca2840790abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6817","nonce":824890163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6819","nonce":824890163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6823","nonce":824890163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6821","nonce":824890163}]},"public_addr":"172.21.15.196:6817/824890163","cluster_addr":"172.21.15.196:6819/824890163","heartbeat_back_addr":"172.21.15.196:6823/824890163","heartbeat_front_addr":"172.21.15.196:6821/824890163","state":["exists","up"]},{"osd":8,"uuid":"10235638-c1f1-41d3-8e24-f8b2344bfeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6817","nonce":3566909552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6819","nonce":3566909552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6823","nonce":3566909552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6821","nonce":3566909552}]},"public_addr":"172.21.15.204:6817/3566909552","cluster_addr":"172.21.15.204:6819/3566909552","heartbeat_back_addr":"172.21.15.204:6823/3566909552","heartbeat_front_addr":"172.21.15.204:6821/3566909552","state":["exists","up"]},{"osd":9,"uuid":"96ce5aa6-e6a9-4d83-8562-589ac94c8cd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6827","nonce":241693472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6829","nonce":241693472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6833","nonce":241693472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6831","nonce":241693472}]},"public_addr":"172.21.15.181:6827/241693472","cluster_addr":"172.21.15.181:6829/241693472","heartbeat_back_addr":"172.21.15.181:6833/241693472","heartbeat_front_addr":"172.21.15.181:6831/241693472","state":["exists","up"]},{"osd":10,"uuid":"d6c9ffab-731c-46d5-88cd-5f21ba5acfa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6825","nonce":3980645995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6827","nonce":3980645995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6831","nonce":3980645995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6829","nonce":3980645995}]},"public_addr":"172.21.15.196:6825/3980645995","cluster_addr":"172.21.15.196:6827/3980645995","heartbeat_back_addr":"172.21.15.196:6831/3980645995","heartbeat_front_addr":"172.21.15.196:6829/3980645995","state":["exists","up"]},{"osd":11,"uuid":"3f58a2d6-bbbb-4e4d-bf83-2c886dd9484b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6825","nonce":488710574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6827","nonce":488710574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6831","nonce":488710574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6829","nonce":488710574}]},"public_addr":"172.21.15.204:6825/488710574","cluster_addr":"172.21.15.204:6827/488710574","heartbeat_back_addr":"172.21.15.204:6831/488710574","heartbeat_front_addr":"172.21.15.204:6829/488710574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:28:09.700091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:28:33.838568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:28:57.265536+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:29:21.720066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:29:45.480447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:30:09.558384+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:30:34.253331+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:30:57.986111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:31:22.470746+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:31:47.548762+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:32:11.285657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:32:36.888004+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6800/2389231557":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/3398276284":"2024-05-04T20:26:06.561713+0000","172.21.15.181:6801/2389231557":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/2327733609":"2024-05-04T20:26:06.561713+0000","172.21.15.181:0/628540759":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/3387163329":"2024-05-04T20:26:06.561713+0000","172.21.15.181:6800/1802984663":"2024-05-04T20:26:06.561713+0000","172.21.15.181:0/67924109":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/3693389657":"2024-05-04T20:26:33.766772+0000","172.21.15.181:6801/1802984663":"2024-05-04T20:26:06.561713+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T20:33:48.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:48.015+0000 7fcc50dbf700 1 -- 172.21.15.181:0/3761565833 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fcc340b2880 msgr2=0x7fcc340b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:48.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:48.015+0000 7fcc50dbf700 1 --2- 172.21.15.181:0/3761565833 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fcc340b2880 0x7fcc340b4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c19ab10 tx=0x7fcc3c00b410 comp rx=0 tx=0).stop 2024-05-03T20:33:48.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:48.015+0000 7fcc50dbf700 1 -- 172.21.15.181:0/3761565833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc4c1087c0 msgr2=0x7fcc4c1999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:48.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:48.015+0000 7fcc50dbf700 1 --2- 172.21.15.181:0/3761565833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc4c1087c0 0x7fcc4c1999d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcc44007ee0 tx=0x7fcc4400d3b0 comp rx=0 tx=0).stop 2024-05-03T20:33:48.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:48.015+0000 7fcc50dbf700 1 -- 172.21.15.181:0/3761565833 shutdown_connections 2024-05-03T20:33:48.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:48.015+0000 7fcc50dbf700 1 --2- 172.21.15.181:0/3761565833 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fcc340b2880 0x7fcc340b4d40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:48.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:48.015+0000 7fcc50dbf700 1 --2- 172.21.15.181:0/3761565833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc4c1087c0 0x7fcc4c1999d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:48.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:48.015+0000 7fcc50dbf700 1 --2- 172.21.15.181:0/3761565833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc4c06be70 0x7fcc4c19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:48.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:48.015+0000 7fcc50dbf700 1 --2- 172.21.15.181:0/3761565833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc4c06c820 0x7fcc4c19fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:48.016 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:48.015+0000 7fcc50dbf700 1 -- 172.21.15.181:0/3761565833 >> 172.21.15.181:0/3761565833 conn(0x7fcc4c0783f0 msgr2=0x7fcc4c107400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:48.016 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:48.015+0000 7fcc50dbf700 1 -- 172.21.15.181:0/3761565833 shutdown_connections 2024-05-03T20:33:48.016 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:48.015+0000 7fcc50dbf700 1 -- 172.21.15.181:0/3761565833 wait complete. 2024-05-03T20:33:48.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:47 smithi204 ceph-mon[33595]: pgmap v287: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 5 op/s 2024-05-03T20:33:48.114 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:47 smithi196 ceph-mon[32858]: pgmap v287: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 5 op/s 2024-05-03T20:33:48.648 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-03T20:33:48.891 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:33:48.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:48 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3761565833' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T20:33:49.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:48 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3761565833' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T20:33:49.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:48 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3761565833' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T20:33:49.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:49 smithi181 ceph-mon[29801]: pgmap v288: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-03T20:33:50.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:49 smithi196 ceph-mon[32858]: pgmap v288: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-03T20:33:50.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:49 smithi204 ceph-mon[33595]: pgmap v288: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-03T20:33:50.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.609+0000 7f4d652e8700 1 -- 172.21.15.181:0/1375921053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d60108850 msgr2=0x7f4d601103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:50.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.609+0000 7f4d652e8700 1 --2- 172.21.15.181:0/1375921053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d60108850 0x7f4d601103d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4d5800b0d0 tx=0x7f4d5800b3e0 comp rx=0 tx=0).stop 2024-05-03T20:33:50.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.609+0000 7f4d652e8700 1 -- 172.21.15.181:0/1375921053 shutdown_connections 2024-05-03T20:33:50.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.609+0000 7f4d652e8700 1 --2- 172.21.15.181:0/1375921053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d60108850 0x7f4d601103d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:50.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.609+0000 7f4d652e8700 1 --2- 172.21.15.181:0/1375921053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d60103750 0x7f4d60103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:50.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.609+0000 7f4d652e8700 1 --2- 172.21.15.181:0/1375921053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d60104100 0x7f4d60108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:50.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.609+0000 7f4d652e8700 1 -- 172.21.15.181:0/1375921053 >> 172.21.15.181:0/1375921053 conn(0x7f4d6007a600 msgr2=0x7f4d6007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:50.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.609+0000 7f4d652e8700 1 -- 172.21.15.181:0/1375921053 shutdown_connections 2024-05-03T20:33:50.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.609+0000 7f4d652e8700 1 -- 172.21.15.181:0/1375921053 wait complete. 2024-05-03T20:33:50.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.609+0000 7f4d652e8700 1 Processor -- start 2024-05-03T20:33:50.611 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.610+0000 7f4d652e8700 1 -- start start 2024-05-03T20:33:50.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.610+0000 7f4d652e8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d60103750 0x7f4d6019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:50.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.610+0000 7f4d652e8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d60104100 0x7f4d6019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:50.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.610+0000 7f4d652e8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d60108850 0x7f4d60199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:50.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.610+0000 7f4d652e8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4d60113690 con 0x7f4d60108850 2024-05-03T20:33:50.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.610+0000 7f4d652e8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4d60113510 con 0x7f4d60104100 2024-05-03T20:33:50.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.610+0000 7f4d652e8700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4d60113810 con 0x7f4d60103750 2024-05-03T20:33:50.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.610+0000 7f4d5f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d60108850 0x7f4d60199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:50.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.610+0000 7f4d5f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d60108850 0x7f4d60199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39386/0 (socket says 172.21.15.181:39386) 2024-05-03T20:33:50.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.610+0000 7f4d5f7fe700 1 -- 172.21.15.181:0/1694295409 learned_addr learned my addr 172.21.15.181:0/1694295409 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:33:50.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.610+0000 7f4d5effd700 1 --2- 172.21.15.181:0/1694295409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d60103750 0x7f4d6019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:50.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.610+0000 7f4d567fc700 1 --2- 172.21.15.181:0/1694295409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d60104100 0x7f4d6019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:50.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.610+0000 7f4d5f7fe700 1 -- 172.21.15.181:0/1694295409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d60103750 msgr2=0x7f4d6019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:50.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.610+0000 7f4d5f7fe700 1 --2- 172.21.15.181:0/1694295409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d60103750 0x7f4d6019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:50.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.610+0000 7f4d5f7fe700 1 -- 172.21.15.181:0/1694295409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d60104100 msgr2=0x7f4d6019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:50.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.610+0000 7f4d5f7fe700 1 --2- 172.21.15.181:0/1694295409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d60104100 0x7f4d6019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:50.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.610+0000 7f4d5f7fe700 1 -- 172.21.15.181:0/1694295409 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6019a030 con 0x7f4d60108850 2024-05-03T20:33:50.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.610+0000 7f4d567fc700 1 --2- 172.21.15.181:0/1694295409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d60104100 0x7f4d6019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:33:50.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.610+0000 7f4d5f7fe700 1 --2- 172.21.15.181:0/1694295409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d60108850 0x7f4d60199850 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009fd0 tx=0x7f4d58007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:50.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.611+0000 7f4d5cff9700 1 -- 172.21.15.181:0/1694295409 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5801f070 con 0x7f4d60108850 2024-05-03T20:33:50.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.611+0000 7f4d5cff9700 1 -- 172.21.15.181:0/1694295409 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d58003c00 con 0x7f4d60108850 2024-05-03T20:33:50.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.611+0000 7f4d652e8700 1 -- 172.21.15.181:0/1694295409 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d6019a2c0 con 0x7f4d60108850 2024-05-03T20:33:50.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.611+0000 7f4d5cff9700 1 -- 172.21.15.181:0/1694295409 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5801b8e0 con 0x7f4d60108850 2024-05-03T20:33:50.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.611+0000 7f4d652e8700 1 -- 172.21.15.181:0/1694295409 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d601a6530 con 0x7f4d60108850 2024-05-03T20:33:50.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.612+0000 7f4d652e8700 1 -- 172.21.15.181:0/1694295409 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d60104be0 con 0x7f4d60108850 2024-05-03T20:33:50.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.613+0000 7f4d5cff9700 1 -- 172.21.15.181:0/1694295409 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f4d58022070 con 0x7f4d60108850 2024-05-03T20:33:50.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.614+0000 7f4d5cff9700 1 --2- 172.21.15.181:0/1694295409 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4d4c0b2880 0x7f4d4c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:50.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.614+0000 7f4d5effd700 1 --2- 172.21.15.181:0/1694295409 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4d4c0b2880 0x7f4d4c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:50.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.614+0000 7f4d5cff9700 1 -- 172.21.15.181:0/1694295409 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f4d5800f020 con 0x7f4d60108850 2024-05-03T20:33:50.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.614+0000 7f4d5effd700 1 --2- 172.21.15.181:0/1694295409 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4d4c0b2880 0x7f4d4c0b4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4d50005ea0 tx=0x7f4d50005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:50.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.617+0000 7f4d5cff9700 1 -- 172.21.15.181:0/1694295409 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4d58029020 con 0x7f4d60108850 2024-05-03T20:33:50.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:50.873+0000 7f4d652e8700 1 -- 172.21.15.181:0/1694295409 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f4d6019ad80 con 0x7f4d60108850 2024-05-03T20:33:51.735 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:51.735+0000 7f4d5cff9700 1 -- 172.21.15.181:0/1694295409 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f4d580ab760 con 0x7f4d60108850 2024-05-03T20:33:51.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:51.738+0000 7f4d652e8700 1 -- 172.21.15.181:0/1694295409 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4d4c0b2880 msgr2=0x7f4d4c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:51.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:51.738+0000 7f4d652e8700 1 --2- 172.21.15.181:0/1694295409 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4d4c0b2880 0x7f4d4c0b4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4d50005ea0 tx=0x7f4d50005e30 comp rx=0 tx=0).stop 2024-05-03T20:33:51.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:51.738+0000 7f4d652e8700 1 -- 172.21.15.181:0/1694295409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d60108850 msgr2=0x7f4d60199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:51.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:51.738+0000 7f4d652e8700 1 --2- 172.21.15.181:0/1694295409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d60108850 0x7f4d60199850 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009fd0 tx=0x7f4d58007ad0 comp rx=0 tx=0).stop 2024-05-03T20:33:51.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:51.738+0000 7f4d652e8700 1 -- 172.21.15.181:0/1694295409 shutdown_connections 2024-05-03T20:33:51.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:51.738+0000 7f4d652e8700 1 --2- 172.21.15.181:0/1694295409 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4d4c0b2880 0x7f4d4c0b4d40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:51.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:51.738+0000 7f4d652e8700 1 --2- 172.21.15.181:0/1694295409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d60108850 0x7f4d60199850 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:51.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:51.738+0000 7f4d652e8700 1 --2- 172.21.15.181:0/1694295409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d60103750 0x7f4d6019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:51.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:51.738+0000 7f4d652e8700 1 --2- 172.21.15.181:0/1694295409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d60104100 0x7f4d6019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:51.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:51.738+0000 7f4d652e8700 1 -- 172.21.15.181:0/1694295409 >> 172.21.15.181:0/1694295409 conn(0x7f4d6007a600 msgr2=0x7f4d60101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:51.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:51.739+0000 7f4d652e8700 1 -- 172.21.15.181:0/1694295409 shutdown_connections 2024-05-03T20:33:51.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:51.739+0000 7f4d652e8700 1 -- 172.21.15.181:0/1694295409 wait complete. 2024-05-03T20:33:52.084 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:51 smithi181 ceph-mon[29801]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-03T20:33:52.084 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:51 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/1694295409' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-03T20:33:52.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:51 smithi196 ceph-mon[32858]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-03T20:33:52.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:51 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/1694295409' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-03T20:33:52.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:51 smithi204 ceph-mon[33595]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-03T20:33:52.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:51 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/1694295409' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-03T20:33:52.434 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-05-03T20:33:52.434 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-05-03T20:33:52.676 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:33:53.020 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:52 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/1694295409' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-03T20:33:53.020 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:52 smithi181 ceph-mon[29801]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-03T20:33:53.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:52 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/1694295409' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-03T20:33:53.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:52 smithi196 ceph-mon[32858]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-03T20:33:53.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:52 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/1694295409' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-03T20:33:53.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:52 smithi204 ceph-mon[33595]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-03T20:33:53.948 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:53 smithi181 ceph-mon[29801]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-03T20:33:54.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:53 smithi196 ceph-mon[32858]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-03T20:33:54.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:53 smithi204 ceph-mon[33595]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-03T20:33:54.168 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.165+0000 7f4f4db77700 1 -- 172.21.15.181:0/409379384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f48108580 msgr2=0x7f4f48108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:54.168 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.165+0000 7f4f4db77700 1 --2- 172.21.15.181:0/409379384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f48108580 0x7f4f48108960 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009990 tx=0x7f4f38009ca0 comp rx=0 tx=0).stop 2024-05-03T20:33:54.168 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.165+0000 7f4f4db77700 1 -- 172.21.15.181:0/409379384 shutdown_connections 2024-05-03T20:33:54.168 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.165+0000 7f4f4db77700 1 --2- 172.21.15.181:0/409379384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f48102820 0x7f4f48110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:54.168 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.165+0000 7f4f4db77700 1 --2- 172.21.15.181:0/409379384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f48108580 0x7f4f48108960 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:54.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.165+0000 7f4f4db77700 1 --2- 172.21.15.181:0/409379384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f48101e60 0x7f4f481022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:54.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.165+0000 7f4f4db77700 1 -- 172.21.15.181:0/409379384 >> 172.21.15.181:0/409379384 conn(0x7f4f480776c0 msgr2=0x7f4f48077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:54.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.165+0000 7f4f4db77700 1 -- 172.21.15.181:0/409379384 shutdown_connections 2024-05-03T20:33:54.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.166+0000 7f4f4db77700 1 -- 172.21.15.181:0/409379384 wait complete. 2024-05-03T20:33:54.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.166+0000 7f4f4db77700 1 Processor -- start 2024-05-03T20:33:54.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.166+0000 7f4f4db77700 1 -- start start 2024-05-03T20:33:54.170 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.166+0000 7f4f4db77700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f48101e60 0x7f4f4819f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:54.170 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.166+0000 7f4f4db77700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f48102820 0x7f4f4819fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:54.170 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.166+0000 7f4f4db77700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f48108580 0x7f4f48199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:54.170 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.166+0000 7f4f4db77700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4f48113770 con 0x7f4f48101e60 2024-05-03T20:33:54.170 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.166+0000 7f4f4db77700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f481135f0 con 0x7f4f48102820 2024-05-03T20:33:54.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.167+0000 7f4f4db77700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4f481138f0 con 0x7f4f48108580 2024-05-03T20:33:54.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.167+0000 7f4f477fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f48101e60 0x7f4f4819f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:54.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.167+0000 7f4f477fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f48101e60 0x7f4f4819f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39410/0 (socket says 172.21.15.181:39410) 2024-05-03T20:33:54.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.167+0000 7f4f477fe700 1 -- 172.21.15.181:0/3846725201 learned_addr learned my addr 172.21.15.181:0/3846725201 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:33:54.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.167+0000 7f4f477fe700 1 -- 172.21.15.181:0/3846725201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f48108580 msgr2=0x7f4f48199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:54.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.167+0000 7f4f477fe700 1 --2- 172.21.15.181:0/3846725201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f48108580 0x7f4f48199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:54.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.167+0000 7f4f477fe700 1 -- 172.21.15.181:0/3846725201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f48102820 msgr2=0x7f4f4819fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:54.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.167+0000 7f4f477fe700 1 --2- 172.21.15.181:0/3846725201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f48102820 0x7f4f4819fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:54.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.167+0000 7f4f477fe700 1 -- 172.21.15.181:0/3846725201 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4819a290 con 0x7f4f48101e60 2024-05-03T20:33:54.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.167+0000 7f4f477fe700 1 --2- 172.21.15.181:0/3846725201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f48101e60 0x7f4f4819f920 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4f38000c00 tx=0x7f4f38003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:54.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.167+0000 7f4f44ff9700 1 -- 172.21.15.181:0/3846725201 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f38027070 con 0x7f4f48101e60 2024-05-03T20:33:54.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.167+0000 7f4f44ff9700 1 -- 172.21.15.181:0/3846725201 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f38004160 con 0x7f4f48101e60 2024-05-03T20:33:54.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.167+0000 7f4f4db77700 1 -- 172.21.15.181:0/3846725201 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4819a520 con 0x7f4f48101e60 2024-05-03T20:33:54.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.168+0000 7f4f44ff9700 1 -- 172.21.15.181:0/3846725201 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f38022db0 con 0x7f4f48101e60 2024-05-03T20:33:54.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.168+0000 7f4f4db77700 1 -- 172.21.15.181:0/3846725201 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f481a67f0 con 0x7f4f48101e60 2024-05-03T20:33:54.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.168+0000 7f4f4db77700 1 -- 172.21.15.181:0/3846725201 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f4806a190 con 0x7f4f48101e60 2024-05-03T20:33:54.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.170+0000 7f4f44ff9700 1 -- 172.21.15.181:0/3846725201 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f4f3802a070 con 0x7f4f48101e60 2024-05-03T20:33:54.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.170+0000 7f4f44ff9700 1 --2- 172.21.15.181:0/3846725201 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4f300b2900 0x7f4f300b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:54.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.170+0000 7f4f46ffd700 1 --2- 172.21.15.181:0/3846725201 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4f300b2900 0x7f4f300b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:54.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.171+0000 7f4f44ff9700 1 -- 172.21.15.181:0/3846725201 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f4f380e2d20 con 0x7f4f48101e60 2024-05-03T20:33:54.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.171+0000 7f4f46ffd700 1 --2- 172.21.15.181:0/3846725201 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4f300b2900 0x7f4f300b4dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4f48077d30 tx=0x7f4f3c006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:54.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.175+0000 7f4f44ff9700 1 -- 172.21.15.181:0/3846725201 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4f38013070 con 0x7f4f48101e60 2024-05-03T20:33:54.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.426+0000 7f4f4db77700 1 -- 172.21.15.181:0/3846725201 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f4f4802e6d0 con 0x7f4f48101e60 2024-05-03T20:33:54.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.449+0000 7f4f44ff9700 1 -- 172.21.15.181:0/3846725201 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f4f380aac20 con 0x7f4f48101e60 2024-05-03T20:33:54.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.452+0000 7f4f4db77700 1 -- 172.21.15.181:0/3846725201 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4f300b2900 msgr2=0x7f4f300b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:54.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.452+0000 7f4f4db77700 1 --2- 172.21.15.181:0/3846725201 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4f300b2900 0x7f4f300b4dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4f48077d30 tx=0x7f4f3c006d20 comp rx=0 tx=0).stop 2024-05-03T20:33:54.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.452+0000 7f4f4db77700 1 -- 172.21.15.181:0/3846725201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f48101e60 msgr2=0x7f4f4819f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:54.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.452+0000 7f4f4db77700 1 --2- 172.21.15.181:0/3846725201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f48101e60 0x7f4f4819f920 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4f38000c00 tx=0x7f4f38003680 comp rx=0 tx=0).stop 2024-05-03T20:33:54.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.452+0000 7f4f4db77700 1 -- 172.21.15.181:0/3846725201 shutdown_connections 2024-05-03T20:33:54.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.452+0000 7f4f4db77700 1 --2- 172.21.15.181:0/3846725201 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4f300b2900 0x7f4f300b4dc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:54.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.452+0000 7f4f4db77700 1 --2- 172.21.15.181:0/3846725201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f48108580 0x7f4f48199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:54.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.452+0000 7f4f4db77700 1 --2- 172.21.15.181:0/3846725201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f48101e60 0x7f4f4819f920 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:54.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.452+0000 7f4f4db77700 1 --2- 172.21.15.181:0/3846725201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f48102820 0x7f4f4819fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:54.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.452+0000 7f4f4db77700 1 -- 172.21.15.181:0/3846725201 >> 172.21.15.181:0/3846725201 conn(0x7f4f480776c0 msgr2=0x7f4f48100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:54.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.452+0000 7f4f4db77700 1 -- 172.21.15.181:0/3846725201 shutdown_connections 2024-05-03T20:33:54.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:54.452+0000 7f4f4db77700 1 -- 172.21.15.181:0/3846725201 wait complete. 2024-05-03T20:33:55.005 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:54 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3846725201' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-03T20:33:55.005 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:54 smithi181 ceph-mon[29801]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-03T20:33:55.005 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:54 smithi181 ceph-mon[29801]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-03T20:33:55.005 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:54 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3846725201' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-03T20:33:55.005 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:54 smithi181 ceph-mon[29801]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-03T20:33:55.005 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:54 smithi181 ceph-mon[29801]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-03T20:33:55.032 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-03T20:33:55.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:54 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3846725201' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-03T20:33:55.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:54 smithi196 ceph-mon[32858]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-03T20:33:55.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:54 smithi196 ceph-mon[32858]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-03T20:33:55.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:54 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3846725201' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-03T20:33:55.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:54 smithi196 ceph-mon[32858]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-03T20:33:55.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:54 smithi196 ceph-mon[32858]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-03T20:33:55.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:54 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3846725201' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-03T20:33:55.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:54 smithi204 ceph-mon[33595]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-03T20:33:55.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:54 smithi204 ceph-mon[33595]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-03T20:33:55.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:54 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3846725201' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-03T20:33:55.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:54 smithi204 ceph-mon[33595]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-03T20:33:55.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:54 smithi204 ceph-mon[33595]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-03T20:33:55.270 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:33:55.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:55 smithi181 ceph-mon[29801]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-03T20:33:55.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:55 smithi181 ceph-mon[29801]: mds.? [v2:172.21.15.204:6838/1644828559,v1:172.21.15.204:6839/1644828559] up:active 2024-05-03T20:33:55.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:55 smithi181 ceph-mon[29801]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-03T20:33:55.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:55 smithi181 ceph-mon[29801]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-03T20:33:55.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:55 smithi181 ceph-mon[29801]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-05-03T20:33:55.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:55 smithi181 ceph-mon[29801]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-05-03T20:33:56.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:55 smithi196 ceph-mon[32858]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-03T20:33:56.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:55 smithi196 ceph-mon[32858]: mds.? [v2:172.21.15.204:6838/1644828559,v1:172.21.15.204:6839/1644828559] up:active 2024-05-03T20:33:56.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:55 smithi196 ceph-mon[32858]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-03T20:33:56.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:55 smithi196 ceph-mon[32858]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-03T20:33:56.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:55 smithi196 ceph-mon[32858]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-05-03T20:33:56.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:55 smithi196 ceph-mon[32858]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-05-03T20:33:56.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:55 smithi204 ceph-mon[33595]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-03T20:33:56.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:55 smithi204 ceph-mon[33595]: mds.? [v2:172.21.15.204:6838/1644828559,v1:172.21.15.204:6839/1644828559] up:active 2024-05-03T20:33:56.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:55 smithi204 ceph-mon[33595]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-03T20:33:56.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:55 smithi204 ceph-mon[33595]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-03T20:33:56.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:55 smithi204 ceph-mon[33595]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-05-03T20:33:56.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:55 smithi204 ceph-mon[33595]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-05-03T20:33:56.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.894+0000 7f4e0b1c5700 1 -- 172.21.15.181:0/834663135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e04106340 msgr2=0x7f4e04106720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:56.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.894+0000 7f4e0b1c5700 1 --2- 172.21.15.181:0/834663135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e04106340 0x7f4e04106720 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4df80099c0 tx=0x7f4df8009cd0 comp rx=0 tx=0).stop 2024-05-03T20:33:56.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.894+0000 7f4e0b1c5700 1 -- 172.21.15.181:0/834663135 shutdown_connections 2024-05-03T20:33:56.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.894+0000 7f4e0b1c5700 1 --2- 172.21.15.181:0/834663135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e04108b70 0x7f4e0410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:56.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.894+0000 7f4e0b1c5700 1 --2- 172.21.15.181:0/834663135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e04106340 0x7f4e04106720 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:56.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.894+0000 7f4e0b1c5700 1 --2- 172.21.15.181:0/834663135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e041081b0 0x7f4e04108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:56.898 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.894+0000 7f4e0b1c5700 1 -- 172.21.15.181:0/834663135 >> 172.21.15.181:0/834663135 conn(0x7f4e040776c0 msgr2=0x7f4e04077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:56.898 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.894+0000 7f4e0b1c5700 1 -- 172.21.15.181:0/834663135 shutdown_connections 2024-05-03T20:33:56.898 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.894+0000 7f4e0b1c5700 1 -- 172.21.15.181:0/834663135 wait complete. 2024-05-03T20:33:56.898 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.895+0000 7f4e0b1c5700 1 Processor -- start 2024-05-03T20:33:56.898 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.895+0000 7f4e0b1c5700 1 -- start start 2024-05-03T20:33:56.898 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.895+0000 7f4e0b1c5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e04106340 0x7f4e041a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:56.899 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.895+0000 7f4e0b1c5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e041081b0 0x7f4e041a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:56.899 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.895+0000 7f4e0b1c5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e04108b70 0x7f4e041a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:56.899 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.895+0000 7f4e0b1c5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4e04112670 con 0x7f4e041081b0 2024-05-03T20:33:56.899 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.895+0000 7f4e0b1c5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4e041124f0 con 0x7f4e04106340 2024-05-03T20:33:56.899 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.895+0000 7f4e0b1c5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4e041127f0 con 0x7f4e04108b70 2024-05-03T20:33:56.899 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.895+0000 7f4e03fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e041081b0 0x7f4e041a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:56.899 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.895+0000 7f4e03fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e041081b0 0x7f4e041a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52562/0 (socket says 172.21.15.181:52562) 2024-05-03T20:33:56.900 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.895+0000 7f4e03fff700 1 -- 172.21.15.181:0/3419190154 learned_addr learned my addr 172.21.15.181:0/3419190154 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:33:56.900 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.895+0000 7f4e08f61700 1 --2- 172.21.15.181:0/3419190154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e04106340 0x7f4e041a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:56.900 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.895+0000 7f4e09762700 1 --2- 172.21.15.181:0/3419190154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e04108b70 0x7f4e041a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:56.900 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.895+0000 7f4e03fff700 1 -- 172.21.15.181:0/3419190154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e04108b70 msgr2=0x7f4e041a4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:56.900 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.895+0000 7f4e03fff700 1 --2- 172.21.15.181:0/3419190154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e04108b70 0x7f4e041a4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:56.900 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.895+0000 7f4e03fff700 1 -- 172.21.15.181:0/3419190154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e04106340 msgr2=0x7f4e041a0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:56.901 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.895+0000 7f4e03fff700 1 --2- 172.21.15.181:0/3419190154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e04106340 0x7f4e041a0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:56.901 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.895+0000 7f4e03fff700 1 -- 172.21.15.181:0/3419190154 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e041a5260 con 0x7f4e041081b0 2024-05-03T20:33:56.901 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.895+0000 7f4e09762700 1 --2- 172.21.15.181:0/3419190154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e04108b70 0x7f4e041a4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:33:56.901 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.895+0000 7f4e03fff700 1 --2- 172.21.15.181:0/3419190154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e041081b0 0x7f4e041a0a60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4df000d950 tx=0x7f4df000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:56.901 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.896+0000 7f4e01ffb700 1 -- 172.21.15.181:0/3419190154 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0016070 con 0x7f4e041081b0 2024-05-03T20:33:56.901 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.896+0000 7f4e01ffb700 1 -- 172.21.15.181:0/3419190154 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4df0009df0 con 0x7f4e041081b0 2024-05-03T20:33:56.901 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.896+0000 7f4e0b1c5700 1 -- 172.21.15.181:0/3419190154 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e041a5550 con 0x7f4e041081b0 2024-05-03T20:33:56.902 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.896+0000 7f4e01ffb700 1 -- 172.21.15.181:0/3419190154 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df000f3c0 con 0x7f4e041081b0 2024-05-03T20:33:56.902 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.896+0000 7f4e0b1c5700 1 -- 172.21.15.181:0/3419190154 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e041acdd0 con 0x7f4e041081b0 2024-05-03T20:33:56.902 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.897+0000 7f4e0b1c5700 1 -- 172.21.15.181:0/3419190154 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e0406a190 con 0x7f4e041081b0 2024-05-03T20:33:56.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.898+0000 7f4e01ffb700 1 -- 172.21.15.181:0/3419190154 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f4df0019070 con 0x7f4e041081b0 2024-05-03T20:33:56.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.899+0000 7f4e01ffb700 1 --2- 172.21.15.181:0/3419190154 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4df40b2880 0x7f4df40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:33:56.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.899+0000 7f4e08f61700 1 --2- 172.21.15.181:0/3419190154 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4df40b2880 0x7f4df40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:33:56.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.899+0000 7f4e01ffb700 1 -- 172.21.15.181:0/3419190154 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f4df00e1d80 con 0x7f4e041081b0 2024-05-03T20:33:56.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.899+0000 7f4e08f61700 1 --2- 172.21.15.181:0/3419190154 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4df40b2880 0x7f4df40b4d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4df8009990 tx=0x7f4df800b790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:33:56.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:56.903+0000 7f4e01ffb700 1 -- 172.21.15.181:0/3419190154 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4df00a9b50 con 0x7f4e041081b0 2024-05-03T20:33:57.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:57.151+0000 7f4e0b1c5700 1 -- 172.21.15.181:0/3419190154 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f4e041acfa0 con 0x7f4e041081b0 2024-05-03T20:33:57.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:57.463+0000 7f4e01ffb700 1 -- 172.21.15.181:0/3419190154 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f4df00a9450 con 0x7f4e041081b0 2024-05-03T20:33:57.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:57.465+0000 7f4e0b1c5700 1 -- 172.21.15.181:0/3419190154 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4df40b2880 msgr2=0x7f4df40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:57.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:57.465+0000 7f4e0b1c5700 1 --2- 172.21.15.181:0/3419190154 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4df40b2880 0x7f4df40b4d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4df8009990 tx=0x7f4df800b790 comp rx=0 tx=0).stop 2024-05-03T20:33:57.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:57.465+0000 7f4e0b1c5700 1 -- 172.21.15.181:0/3419190154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e041081b0 msgr2=0x7f4e041a0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:33:57.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:57.465+0000 7f4e0b1c5700 1 --2- 172.21.15.181:0/3419190154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e041081b0 0x7f4e041a0a60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4df000d950 tx=0x7f4df000dc60 comp rx=0 tx=0).stop 2024-05-03T20:33:57.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:57.466+0000 7f4e0b1c5700 1 -- 172.21.15.181:0/3419190154 shutdown_connections 2024-05-03T20:33:57.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:57.466+0000 7f4e0b1c5700 1 --2- 172.21.15.181:0/3419190154 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4df40b2880 0x7f4df40b4d40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:57.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:57.466+0000 7f4e0b1c5700 1 --2- 172.21.15.181:0/3419190154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e04108b70 0x7f4e041a4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:57.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:57.466+0000 7f4e0b1c5700 1 --2- 172.21.15.181:0/3419190154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e04106340 0x7f4e041a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:57.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:57.466+0000 7f4e0b1c5700 1 --2- 172.21.15.181:0/3419190154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e041081b0 0x7f4e041a0a60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:33:57.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:57.466+0000 7f4e0b1c5700 1 -- 172.21.15.181:0/3419190154 >> 172.21.15.181:0/3419190154 conn(0x7f4e040776c0 msgr2=0x7f4e0410b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:33:57.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:57.466+0000 7f4e0b1c5700 1 -- 172.21.15.181:0/3419190154 shutdown_connections 2024-05-03T20:33:57.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:33:57.466+0000 7f4e0b1c5700 1 -- 172.21.15.181:0/3419190154 wait complete. 2024-05-03T20:33:57.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:57 smithi196 ceph-mon[32858]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 4 op/s 2024-05-03T20:33:57.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:57 smithi196 ceph-mon[32858]: mds.? [v2:172.21.15.196:6836/1529420259,v1:172.21.15.196:6837/1529420259] up:active 2024-05-03T20:33:57.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:57 smithi196 ceph-mon[32858]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-03T20:33:57.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:57 smithi196 ceph-mon[32858]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-03T20:33:57.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:57 smithi196 ceph-mon[32858]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-05-03T20:33:57.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:57 smithi196 ceph-mon[32858]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-05-03T20:33:57.607 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:57 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3419190154' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-03T20:33:57.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:57 smithi181 ceph-mon[29801]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 4 op/s 2024-05-03T20:33:57.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:57 smithi181 ceph-mon[29801]: mds.? [v2:172.21.15.196:6836/1529420259,v1:172.21.15.196:6837/1529420259] up:active 2024-05-03T20:33:57.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:57 smithi181 ceph-mon[29801]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-03T20:33:57.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:57 smithi181 ceph-mon[29801]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-03T20:33:57.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:57 smithi181 ceph-mon[29801]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-05-03T20:33:57.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:57 smithi181 ceph-mon[29801]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-05-03T20:33:57.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:57 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3419190154' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-03T20:33:57.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:57 smithi204 ceph-mon[33595]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 4 op/s 2024-05-03T20:33:57.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:57 smithi204 ceph-mon[33595]: mds.? [v2:172.21.15.196:6836/1529420259,v1:172.21.15.196:6837/1529420259] up:active 2024-05-03T20:33:57.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:57 smithi204 ceph-mon[33595]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-03T20:33:57.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:57 smithi204 ceph-mon[33595]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-03T20:33:57.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:57 smithi204 ceph-mon[33595]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-05-03T20:33:57.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:57 smithi204 ceph-mon[33595]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-05-03T20:33:57.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:57 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3419190154' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-03T20:33:58.208 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-03T20:33:58.456 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:33:58.748 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:58 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3419190154' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-03T20:33:58.748 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:58 smithi181 ceph-mon[29801]: mds.? [v2:172.21.15.204:6836/3006728559,v1:172.21.15.204:6837/3006728559] up:active 2024-05-03T20:33:58.748 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:58 smithi181 ceph-mon[29801]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-03T20:33:58.748 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:58 smithi181 ceph-mon[29801]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-03T20:33:58.748 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:58 smithi181 ceph-mon[29801]: Cluster is now healthy 2024-05-03T20:33:58.748 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:58 smithi181 ceph-mon[29801]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-05-03T20:33:58.748 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:58 smithi181 ceph-mon[29801]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-03T20:33:58.748 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:58 smithi181 ceph-mon[29801]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-05-03T20:33:58.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:58 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3419190154' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-03T20:33:58.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:58 smithi196 ceph-mon[32858]: mds.? [v2:172.21.15.204:6836/3006728559,v1:172.21.15.204:6837/3006728559] up:active 2024-05-03T20:33:58.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:58 smithi196 ceph-mon[32858]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-03T20:33:58.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:58 smithi196 ceph-mon[32858]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-03T20:33:58.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:58 smithi196 ceph-mon[32858]: Cluster is now healthy 2024-05-03T20:33:58.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:58 smithi196 ceph-mon[32858]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-05-03T20:33:58.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:58 smithi196 ceph-mon[32858]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-03T20:33:58.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:58 smithi196 ceph-mon[32858]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-05-03T20:33:58.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:58 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3419190154' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-03T20:33:58.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:58 smithi204 ceph-mon[33595]: mds.? [v2:172.21.15.204:6836/3006728559,v1:172.21.15.204:6837/3006728559] up:active 2024-05-03T20:33:58.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:58 smithi204 ceph-mon[33595]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-03T20:33:58.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:58 smithi204 ceph-mon[33595]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-03T20:33:58.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:58 smithi204 ceph-mon[33595]: Cluster is now healthy 2024-05-03T20:33:58.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:58 smithi204 ceph-mon[33595]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-05-03T20:33:58.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:58 smithi204 ceph-mon[33595]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-03T20:33:58.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:58 smithi204 ceph-mon[33595]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-05-03T20:33:59.628 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:59 smithi181 ceph-mon[29801]: pgmap v293: 97 pgs: 97 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 6 op/s 2024-05-03T20:33:59.628 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:59 smithi181 ceph-mon[29801]: mds.? [v2:172.21.15.204:6834/3870850287,v1:172.21.15.204:6835/3870850287] up:active 2024-05-03T20:33:59.628 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:59 smithi181 ceph-mon[29801]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-03T20:33:59.628 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:33:59 smithi181 ceph-mon[29801]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-03T20:33:59.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:59 smithi196 ceph-mon[32858]: pgmap v293: 97 pgs: 97 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 6 op/s 2024-05-03T20:33:59.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:59 smithi196 ceph-mon[32858]: mds.? [v2:172.21.15.204:6834/3870850287,v1:172.21.15.204:6835/3870850287] up:active 2024-05-03T20:33:59.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:59 smithi196 ceph-mon[32858]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-03T20:33:59.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:33:59 smithi196 ceph-mon[32858]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-03T20:33:59.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:59 smithi204 ceph-mon[33595]: pgmap v293: 97 pgs: 97 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 6 op/s 2024-05-03T20:33:59.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:59 smithi204 ceph-mon[33595]: mds.? [v2:172.21.15.204:6834/3870850287,v1:172.21.15.204:6835/3870850287] up:active 2024-05-03T20:33:59.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:59 smithi204 ceph-mon[33595]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-03T20:33:59.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:33:59 smithi204 ceph-mon[33595]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-03T20:34:00.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.093+0000 7f1a891f8700 1 -- 172.21.15.181:0/1037620449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a84102d60 msgr2=0x7f1a8410f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:00.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.093+0000 7f1a891f8700 1 --2- 172.21.15.181:0/1037620449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a84102d60 0x7f1a8410f250 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c00b0a0 tx=0x7f1a7c00b3b0 comp rx=0 tx=0).stop 2024-05-03T20:34:00.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.093+0000 7f1a891f8700 1 -- 172.21.15.181:0/1037620449 shutdown_connections 2024-05-03T20:34:00.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.093+0000 7f1a891f8700 1 --2- 172.21.15.181:0/1037620449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a84102d60 0x7f1a8410f250 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:00.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.093+0000 7f1a891f8700 1 --2- 172.21.15.181:0/1037620449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a84108410 0x7f1a841087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:00.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.093+0000 7f1a891f8700 1 --2- 172.21.15.181:0/1037620449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a841023a0 0x7f1a84102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:00.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.093+0000 7f1a891f8700 1 -- 172.21.15.181:0/1037620449 >> 172.21.15.181:0/1037620449 conn(0x7f1a84079350 msgr2=0x7f1a84079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:00.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.093+0000 7f1a891f8700 1 -- 172.21.15.181:0/1037620449 shutdown_connections 2024-05-03T20:34:00.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.093+0000 7f1a891f8700 1 -- 172.21.15.181:0/1037620449 wait complete. 2024-05-03T20:34:00.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.094+0000 7f1a891f8700 1 Processor -- start 2024-05-03T20:34:00.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.094+0000 7f1a891f8700 1 -- start start 2024-05-03T20:34:00.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.094+0000 7f1a891f8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a841023a0 0x7f1a84104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:00.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.094+0000 7f1a891f8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a84102d60 0x7f1a841045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:00.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.094+0000 7f1a891f8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a84108410 0x7f1a84104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:00.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.094+0000 7f1a891f8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1a841125d0 con 0x7f1a84108410 2024-05-03T20:34:00.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.094+0000 7f1a891f8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1a84112450 con 0x7f1a84102d60 2024-05-03T20:34:00.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.094+0000 7f1a891f8700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1a84112750 con 0x7f1a841023a0 2024-05-03T20:34:00.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.094+0000 7f1a8359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a84108410 0x7f1a84104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:00.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.094+0000 7f1a8359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a84108410 0x7f1a84104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52570/0 (socket says 172.21.15.181:52570) 2024-05-03T20:34:00.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.094+0000 7f1a8359e700 1 -- 172.21.15.181:0/365352559 learned_addr learned my addr 172.21.15.181:0/365352559 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:34:00.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.095+0000 7f1a8259c700 1 --2- 172.21.15.181:0/365352559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a84102d60 0x7f1a841045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:00.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.095+0000 7f1a82d9d700 1 --2- 172.21.15.181:0/365352559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a841023a0 0x7f1a84104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:00.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.095+0000 7f1a8359e700 1 -- 172.21.15.181:0/365352559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a841023a0 msgr2=0x7f1a84104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:00.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.095+0000 7f1a8359e700 1 --2- 172.21.15.181:0/365352559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a841023a0 0x7f1a84104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:00.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.095+0000 7f1a8359e700 1 -- 172.21.15.181:0/365352559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a84102d60 msgr2=0x7f1a841045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:00.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.095+0000 7f1a8359e700 1 --2- 172.21.15.181:0/365352559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a84102d60 0x7f1a841045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:00.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.095+0000 7f1a8359e700 1 -- 172.21.15.181:0/365352559 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a841a80a0 con 0x7f1a84108410 2024-05-03T20:34:00.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.095+0000 7f1a8359e700 1 --2- 172.21.15.181:0/365352559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a84108410 0x7f1a84104b00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c0062a0 tx=0x7f1a7c004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:00.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.095+0000 7f1a6bfff700 1 -- 172.21.15.181:0/365352559 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c021070 con 0x7f1a84108410 2024-05-03T20:34:00.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.095+0000 7f1a891f8700 1 -- 172.21.15.181:0/365352559 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a841a8330 con 0x7f1a84108410 2024-05-03T20:34:00.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.095+0000 7f1a6bfff700 1 -- 172.21.15.181:0/365352559 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a7c007d00 con 0x7f1a84108410 2024-05-03T20:34:00.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.095+0000 7f1a6bfff700 1 -- 172.21.15.181:0/365352559 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c01d450 con 0x7f1a84108410 2024-05-03T20:34:00.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.095+0000 7f1a891f8700 1 -- 172.21.15.181:0/365352559 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a841a8830 con 0x7f1a84108410 2024-05-03T20:34:00.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.096+0000 7f1a891f8700 1 -- 172.21.15.181:0/365352559 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a8406a190 con 0x7f1a84108410 2024-05-03T20:34:00.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.097+0000 7f1a6bfff700 1 -- 172.21.15.181:0/365352559 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f1a7c004020 con 0x7f1a84108410 2024-05-03T20:34:00.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.098+0000 7f1a6bfff700 1 --2- 172.21.15.181:0/365352559 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f1a6c0b2880 0x7f1a6c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:00.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.098+0000 7f1a82d9d700 1 --2- 172.21.15.181:0/365352559 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f1a6c0b2880 0x7f1a6c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:00.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.099+0000 7f1a6bfff700 1 -- 172.21.15.181:0/365352559 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f1a7c00f020 con 0x7f1a84108410 2024-05-03T20:34:00.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.099+0000 7f1a82d9d700 1 --2- 172.21.15.181:0/365352559 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f1a6c0b2880 0x7f1a6c0b4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1a78005950 tx=0x7f1a78005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:00.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.102+0000 7f1a6bfff700 1 -- 172.21.15.181:0/365352559 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1a7c02a020 con 0x7f1a84108410 2024-05-03T20:34:00.357 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.357+0000 7f1a891f8700 1 -- 172.21.15.181:0/365352559 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f1a841a8b50 con 0x7f1a84108410 2024-05-03T20:34:00.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.488+0000 7f1a6bfff700 1 -- 172.21.15.181:0/365352559 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v27) ==== 115+0+0 (secure 0 0 0) 0x7f1a7c0abae0 con 0x7f1a84108410 2024-05-03T20:34:00.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.491+0000 7f1a891f8700 1 -- 172.21.15.181:0/365352559 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f1a6c0b2880 msgr2=0x7f1a6c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:00.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.491+0000 7f1a891f8700 1 --2- 172.21.15.181:0/365352559 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f1a6c0b2880 0x7f1a6c0b4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1a78005950 tx=0x7f1a78005eb0 comp rx=0 tx=0).stop 2024-05-03T20:34:00.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.491+0000 7f1a891f8700 1 -- 172.21.15.181:0/365352559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a84108410 msgr2=0x7f1a84104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:00.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.491+0000 7f1a891f8700 1 --2- 172.21.15.181:0/365352559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a84108410 0x7f1a84104b00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c0062a0 tx=0x7f1a7c004ed0 comp rx=0 tx=0).stop 2024-05-03T20:34:00.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.491+0000 7f1a891f8700 1 -- 172.21.15.181:0/365352559 shutdown_connections 2024-05-03T20:34:00.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.491+0000 7f1a891f8700 1 --2- 172.21.15.181:0/365352559 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f1a6c0b2880 0x7f1a6c0b4d40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:00.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.491+0000 7f1a891f8700 1 --2- 172.21.15.181:0/365352559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a84108410 0x7f1a84104b00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:00.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.491+0000 7f1a891f8700 1 --2- 172.21.15.181:0/365352559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a841023a0 0x7f1a84104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:00.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.491+0000 7f1a891f8700 1 --2- 172.21.15.181:0/365352559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a84102d60 0x7f1a841045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:00.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.491+0000 7f1a891f8700 1 -- 172.21.15.181:0/365352559 >> 172.21.15.181:0/365352559 conn(0x7f1a84079350 msgr2=0x7f1a84101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:00.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.491+0000 7f1a891f8700 1 -- 172.21.15.181:0/365352559 shutdown_connections 2024-05-03T20:34:00.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:00.491+0000 7f1a891f8700 1 -- 172.21.15.181:0/365352559 wait complete. 2024-05-03T20:34:00.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:00 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/365352559' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-03T20:34:00.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:00 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/365352559' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-03T20:34:00.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:00 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/365352559' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-03T20:34:01.205 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-05-03T20:34:01.205 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-03T20:34:01.445 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:34:01.797 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:01 smithi181 ceph-mon[29801]: pgmap v294: 97 pgs: 97 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-05-03T20:34:01.797 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:01 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/365352559' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-03T20:34:01.797 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:01 smithi181 ceph-mon[29801]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-03T20:34:01.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:01 smithi196 ceph-mon[32858]: pgmap v294: 97 pgs: 97 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-05-03T20:34:01.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:01 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/365352559' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-03T20:34:01.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:01 smithi196 ceph-mon[32858]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-03T20:34:01.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:01 smithi204 ceph-mon[33595]: pgmap v294: 97 pgs: 97 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-05-03T20:34:01.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:01 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/365352559' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-03T20:34:01.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:01 smithi204 ceph-mon[33595]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-03T20:34:03.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.049+0000 7f8ec1036700 1 -- 172.21.15.181:0/1001160932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ebc1023d0 msgr2=0x7f8ebc102850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:03.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.049+0000 7f8ec1036700 1 --2- 172.21.15.181:0/1001160932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ebc1023d0 0x7f8ebc102850 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0009990 tx=0x7f8eb0009ca0 comp rx=0 tx=0).stop 2024-05-03T20:34:03.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.049+0000 7f8ec1036700 1 -- 172.21.15.181:0/1001160932 shutdown_connections 2024-05-03T20:34:03.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.049+0000 7f8ec1036700 1 --2- 172.21.15.181:0/1001160932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ebc102d90 0x7f8ebc10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:03.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.049+0000 7f8ec1036700 1 --2- 172.21.15.181:0/1001160932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ebc108450 0x7f8ebc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:03.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.049+0000 7f8ec1036700 1 --2- 172.21.15.181:0/1001160932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ebc1023d0 0x7f8ebc102850 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:03.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.049+0000 7f8ec1036700 1 -- 172.21.15.181:0/1001160932 >> 172.21.15.181:0/1001160932 conn(0x7f8ebc078410 msgr2=0x7f8ebc078820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:03.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.049+0000 7f8ec1036700 1 -- 172.21.15.181:0/1001160932 shutdown_connections 2024-05-03T20:34:03.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.049+0000 7f8ec1036700 1 -- 172.21.15.181:0/1001160932 wait complete. 2024-05-03T20:34:03.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.050+0000 7f8ec1036700 1 Processor -- start 2024-05-03T20:34:03.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.050+0000 7f8ec1036700 1 -- start start 2024-05-03T20:34:03.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.051+0000 7f8ec1036700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ebc1023d0 0x7f8ebc19be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:03.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.051+0000 7f8ec1036700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ebc102d90 0x7f8ebc19c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:03.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.051+0000 7f8ec1036700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ebc108450 0x7f8ebc1a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:03.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.051+0000 7f8ec1036700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8ebc1125a0 con 0x7f8ebc1023d0 2024-05-03T20:34:03.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.051+0000 7f8ec1036700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ebc112420 con 0x7f8ebc102d90 2024-05-03T20:34:03.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.051+0000 7f8ec1036700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8ebc112720 con 0x7f8ebc108450 2024-05-03T20:34:03.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.051+0000 7f8ebad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ebc1023d0 0x7f8ebc19be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:03.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.051+0000 7f8ebad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ebc1023d0 0x7f8ebc19be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52598/0 (socket says 172.21.15.181:52598) 2024-05-03T20:34:03.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.051+0000 7f8ebad9d700 1 -- 172.21.15.181:0/2799679795 learned_addr learned my addr 172.21.15.181:0/2799679795 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:34:03.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.051+0000 7f8eba59c700 1 --2- 172.21.15.181:0/2799679795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ebc102d90 0x7f8ebc19c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:03.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.051+0000 7f8ebb59e700 1 --2- 172.21.15.181:0/2799679795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ebc108450 0x7f8ebc1a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:03.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.051+0000 7f8ebad9d700 1 -- 172.21.15.181:0/2799679795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ebc108450 msgr2=0x7f8ebc1a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:03.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.051+0000 7f8ebad9d700 1 --2- 172.21.15.181:0/2799679795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ebc108450 0x7f8ebc1a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:03.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.051+0000 7f8ebad9d700 1 -- 172.21.15.181:0/2799679795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ebc102d90 msgr2=0x7f8ebc19c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:03.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.051+0000 7f8ebad9d700 1 --2- 172.21.15.181:0/2799679795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ebc102d90 0x7f8ebc19c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:03.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.051+0000 7f8ebad9d700 1 -- 172.21.15.181:0/2799679795 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ebc1a0e30 con 0x7f8ebc1023d0 2024-05-03T20:34:03.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.051+0000 7f8ebb59e700 1 --2- 172.21.15.181:0/2799679795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ebc108450 0x7f8ebc1a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:34:03.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.051+0000 7f8ebad9d700 1 --2- 172.21.15.181:0/2799679795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ebc1023d0 0x7f8ebc19be70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00b530 tx=0x7f8eac00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:03.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.051+0000 7f8ea3fff700 1 -- 172.21.15.181:0/2799679795 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac015020 con 0x7f8ebc1023d0 2024-05-03T20:34:03.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.051+0000 7f8ec1036700 1 -- 172.21.15.181:0/2799679795 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ebc1a1120 con 0x7f8ebc1023d0 2024-05-03T20:34:03.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.051+0000 7f8ea3fff700 1 -- 172.21.15.181:0/2799679795 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eac011ce0 con 0x7f8ebc1023d0 2024-05-03T20:34:03.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.051+0000 7f8ea3fff700 1 -- 172.21.15.181:0/2799679795 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac010d20 con 0x7f8ebc1023d0 2024-05-03T20:34:03.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.052+0000 7f8ec1036700 1 -- 172.21.15.181:0/2799679795 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ebc1a89a0 con 0x7f8ebc1023d0 2024-05-03T20:34:03.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.052+0000 7f8ec1036700 1 -- 172.21.15.181:0/2799679795 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ebc103ac0 con 0x7f8ebc1023d0 2024-05-03T20:34:03.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.054+0000 7f8ea3fff700 1 -- 172.21.15.181:0/2799679795 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f8eac018020 con 0x7f8ebc1023d0 2024-05-03T20:34:03.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.054+0000 7f8ea3fff700 1 --2- 172.21.15.181:0/2799679795 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8ea40b2880 0x7f8ea40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:03.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.054+0000 7f8eba59c700 1 --2- 172.21.15.181:0/2799679795 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8ea40b2880 0x7f8ea40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:03.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.054+0000 7f8ea3fff700 1 -- 172.21.15.181:0/2799679795 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f8eac0e1ed0 con 0x7f8ebc1023d0 2024-05-03T20:34:03.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.055+0000 7f8eba59c700 1 --2- 172.21.15.181:0/2799679795 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8ea40b2880 0x7f8ea40b4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0006010 tx=0x7f8eb001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:03.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.058+0000 7f8ea3fff700 1 -- 172.21.15.181:0/2799679795 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8eac01f020 con 0x7f8ebc1023d0 2024-05-03T20:34:03.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.298+0000 7f8ec1036700 1 -- 172.21.15.181:0/2799679795 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f8ebc06c3c0 con 0x7f8ea40b2880 2024-05-03T20:34:03.326 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.328+0000 7f8ea3fff700 1 -- 172.21.15.181:0/2799679795 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8ebc06c3c0 con 0x7f8ea40b2880 2024-05-03T20:34:03.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.330+0000 7f8ec1036700 1 -- 172.21.15.181:0/2799679795 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8ea40b2880 msgr2=0x7f8ea40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:03.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.330+0000 7f8ec1036700 1 --2- 172.21.15.181:0/2799679795 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8ea40b2880 0x7f8ea40b4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0006010 tx=0x7f8eb001a040 comp rx=0 tx=0).stop 2024-05-03T20:34:03.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.330+0000 7f8ec1036700 1 -- 172.21.15.181:0/2799679795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ebc1023d0 msgr2=0x7f8ebc19be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:03.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.330+0000 7f8ec1036700 1 --2- 172.21.15.181:0/2799679795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ebc1023d0 0x7f8ebc19be70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00b530 tx=0x7f8eac00b8f0 comp rx=0 tx=0).stop 2024-05-03T20:34:03.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.330+0000 7f8ec1036700 1 -- 172.21.15.181:0/2799679795 shutdown_connections 2024-05-03T20:34:03.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.330+0000 7f8ec1036700 1 --2- 172.21.15.181:0/2799679795 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8ea40b2880 0x7f8ea40b4d40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:03.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.330+0000 7f8ec1036700 1 --2- 172.21.15.181:0/2799679795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ebc108450 0x7f8ebc1a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:03.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.330+0000 7f8ec1036700 1 --2- 172.21.15.181:0/2799679795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ebc1023d0 0x7f8ebc19be70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:03.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.330+0000 7f8ec1036700 1 --2- 172.21.15.181:0/2799679795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ebc102d90 0x7f8ebc19c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:03.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.330+0000 7f8ec1036700 1 -- 172.21.15.181:0/2799679795 >> 172.21.15.181:0/2799679795 conn(0x7f8ebc078410 msgr2=0x7f8ebc06bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:03.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.330+0000 7f8ec1036700 1 -- 172.21.15.181:0/2799679795 shutdown_connections 2024-05-03T20:34:03.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:03.330+0000 7f8ec1036700 1 -- 172.21.15.181:0/2799679795 wait complete. 2024-05-03T20:34:03.609 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:03 smithi181 ceph-mon[29801]: pgmap v295: 97 pgs: 97 active+clean; 584 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.9 KiB/s wr, 15 op/s 2024-05-03T20:34:03.742 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:34:03 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:34:03.303+0000 7f4cc46bc700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T20:34:03.742 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:34:03 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:34:03.303+0000 7f4cc46bc700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T20:34:03.742 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:34:03 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:34:03.303+0000 7f4cc46bc700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T20:34:03.742 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:34:03 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:34:03.303+0000 7f4cc46bc700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T20:34:03.743 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:34:03 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:34:03.303+0000 7f4cc46bc700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T20:34:03.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:03 smithi196 ceph-mon[32858]: pgmap v295: 97 pgs: 97 active+clean; 584 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.9 KiB/s wr, 15 op/s 2024-05-03T20:34:03.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:03 smithi204 ceph-mon[33595]: pgmap v295: 97 pgs: 97 active+clean; 584 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.9 KiB/s wr, 15 op/s 2024-05-03T20:34:03.954 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-03T20:34:04.196 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:34:04.570 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:04 smithi181 ceph-mon[29801]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:34:04.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:04 smithi196 ceph-mon[32858]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:34:04.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:04 smithi204 ceph-mon[33595]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:34:05.613 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:05 smithi181 ceph-mon[29801]: pgmap v296: 97 pgs: 97 active+clean; 584 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-05-03T20:34:05.613 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:05 smithi181 ceph-mon[29801]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-03T20:34:05.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.629+0000 7f931ce7d700 1 -- 172.21.15.181:0/2438687614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9318103040 msgr2=0x7f9318107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:05.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.629+0000 7f931ce7d700 1 --2- 172.21.15.181:0/2438687614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9318103040 0x7f9318107000 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f930c0099c0 tx=0x7f930c009cd0 comp rx=0 tx=0).stop 2024-05-03T20:34:05.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.629+0000 7f931ce7d700 1 -- 172.21.15.181:0/2438687614 shutdown_connections 2024-05-03T20:34:05.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.629+0000 7f931ce7d700 1 --2- 172.21.15.181:0/2438687614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93181076e0 0x7f931810f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:05.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.629+0000 7f931ce7d700 1 --2- 172.21.15.181:0/2438687614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9318102690 0x7f9318102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:05.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.629+0000 7f931ce7d700 1 --2- 172.21.15.181:0/2438687614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9318103040 0x7f9318107000 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:05.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.629+0000 7f931ce7d700 1 -- 172.21.15.181:0/2438687614 >> 172.21.15.181:0/2438687614 conn(0x7f93180783f0 msgr2=0x7f9318078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:05.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.629+0000 7f931ce7d700 1 -- 172.21.15.181:0/2438687614 shutdown_connections 2024-05-03T20:34:05.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.629+0000 7f931ce7d700 1 -- 172.21.15.181:0/2438687614 wait complete. 2024-05-03T20:34:05.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.630+0000 7f931ce7d700 1 Processor -- start 2024-05-03T20:34:05.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.630+0000 7f931ce7d700 1 -- start start 2024-05-03T20:34:05.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.630+0000 7f931ce7d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9318102690 0x7f931819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:05.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.630+0000 7f931ce7d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9318103040 0x7f931819af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:05.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.630+0000 7f931ce7d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93181076e0 0x7f93181a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:05.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.630+0000 7f931ce7d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f93181125d0 con 0x7f9318103040 2024-05-03T20:34:05.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.630+0000 7f931ce7d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9318112450 con 0x7f9318102690 2024-05-03T20:34:05.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.630+0000 7f931ce7d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9318112750 con 0x7f93181076e0 2024-05-03T20:34:05.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.630+0000 7f9315d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9318103040 0x7f931819af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:05.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.630+0000 7f9315d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9318103040 0x7f931819af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54374/0 (socket says 172.21.15.181:54374) 2024-05-03T20:34:05.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.630+0000 7f9315d9b700 1 -- 172.21.15.181:0/3765512938 learned_addr learned my addr 172.21.15.181:0/3765512938 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:34:05.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.630+0000 7f9316d9d700 1 --2- 172.21.15.181:0/3765512938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93181076e0 0x7f93181a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:05.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.630+0000 7f931659c700 1 --2- 172.21.15.181:0/3765512938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9318102690 0x7f931819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:05.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.630+0000 7f9315d9b700 1 -- 172.21.15.181:0/3765512938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93181076e0 msgr2=0x7f93181a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:05.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.630+0000 7f9315d9b700 1 --2- 172.21.15.181:0/3765512938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93181076e0 0x7f93181a5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:05.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.630+0000 7f9315d9b700 1 -- 172.21.15.181:0/3765512938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9318102690 msgr2=0x7f931819aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:05.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.630+0000 7f9315d9b700 1 --2- 172.21.15.181:0/3765512938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9318102690 0x7f931819aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:05.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.630+0000 7f9315d9b700 1 -- 172.21.15.181:0/3765512938 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93181a5ff0 con 0x7f9318103040 2024-05-03T20:34:05.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.631+0000 7f9315d9b700 1 --2- 172.21.15.181:0/3765512938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9318103040 0x7f931819af70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f930c009f50 tx=0x7f930c0037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:05.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.631+0000 7f93077fe700 1 -- 172.21.15.181:0/3765512938 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c027070 con 0x7f9318103040 2024-05-03T20:34:05.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.631+0000 7f93077fe700 1 -- 172.21.15.181:0/3765512938 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f930c003fa0 con 0x7f9318103040 2024-05-03T20:34:05.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.631+0000 7f931ce7d700 1 -- 172.21.15.181:0/3765512938 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93181a61c0 con 0x7f9318103040 2024-05-03T20:34:05.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.631+0000 7f93077fe700 1 -- 172.21.15.181:0/3765512938 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c022d20 con 0x7f9318103040 2024-05-03T20:34:05.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.631+0000 7f931ce7d700 1 -- 172.21.15.181:0/3765512938 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93181a6610 con 0x7f9318103040 2024-05-03T20:34:05.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.632+0000 7f931ce7d700 1 -- 172.21.15.181:0/3765512938 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f931806a190 con 0x7f9318103040 2024-05-03T20:34:05.633 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.633+0000 7f93077fe700 1 -- 172.21.15.181:0/3765512938 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f930c02a070 con 0x7f9318103040 2024-05-03T20:34:05.633 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.633+0000 7f93077fe700 1 --2- 172.21.15.181:0/3765512938 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f93000b2910 0x7f93000b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:05.633 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.634+0000 7f931659c700 1 --2- 172.21.15.181:0/3765512938 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f93000b2910 0x7f93000b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:05.633 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.634+0000 7f93077fe700 1 -- 172.21.15.181:0/3765512938 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f930c0e3060 con 0x7f9318103040 2024-05-03T20:34:05.633 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.634+0000 7f931659c700 1 --2- 172.21.15.181:0/3765512938 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f93000b2910 0x7f93000b4dd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9308009a10 tx=0x7f9308009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:05.636 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.638+0000 7f93077fe700 1 -- 172.21.15.181:0/3765512938 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f930c013070 con 0x7f9318103040 2024-05-03T20:34:05.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:05 smithi196 ceph-mon[32858]: pgmap v296: 97 pgs: 97 active+clean; 584 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-05-03T20:34:05.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:05 smithi196 ceph-mon[32858]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-03T20:34:05.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:05 smithi204 ceph-mon[33595]: pgmap v296: 97 pgs: 97 active+clean; 584 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-05-03T20:34:05.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:05 smithi204 ceph-mon[33595]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-03T20:34:05.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.876+0000 7f931ce7d700 1 -- 172.21.15.181:0/3765512938 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f93180638e0 con 0x7f93000b2910 2024-05-03T20:34:05.900 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.901+0000 7f93077fe700 1 -- 172.21.15.181:0/3765512938 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f93180638e0 con 0x7f93000b2910 2024-05-03T20:34:05.902 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.903+0000 7f931ce7d700 1 -- 172.21.15.181:0/3765512938 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f93000b2910 msgr2=0x7f93000b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:05.902 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.903+0000 7f931ce7d700 1 --2- 172.21.15.181:0/3765512938 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f93000b2910 0x7f93000b4dd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9308009a10 tx=0x7f9308009190 comp rx=0 tx=0).stop 2024-05-03T20:34:05.902 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.903+0000 7f931ce7d700 1 -- 172.21.15.181:0/3765512938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9318103040 msgr2=0x7f931819af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:05.902 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.903+0000 7f931ce7d700 1 --2- 172.21.15.181:0/3765512938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9318103040 0x7f931819af70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f930c009f50 tx=0x7f930c0037c0 comp rx=0 tx=0).stop 2024-05-03T20:34:05.902 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.903+0000 7f931ce7d700 1 -- 172.21.15.181:0/3765512938 shutdown_connections 2024-05-03T20:34:05.902 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.903+0000 7f931ce7d700 1 --2- 172.21.15.181:0/3765512938 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f93000b2910 0x7f93000b4dd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:05.902 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.903+0000 7f931ce7d700 1 --2- 172.21.15.181:0/3765512938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93181076e0 0x7f93181a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:05.902 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.903+0000 7f931ce7d700 1 --2- 172.21.15.181:0/3765512938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9318102690 0x7f931819aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:05.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.903+0000 7f931ce7d700 1 --2- 172.21.15.181:0/3765512938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9318103040 0x7f931819af70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:05.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.903+0000 7f931ce7d700 1 -- 172.21.15.181:0/3765512938 >> 172.21.15.181:0/3765512938 conn(0x7f93180783f0 msgr2=0x7f93181088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:05.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.904+0000 7f931ce7d700 1 -- 172.21.15.181:0/3765512938 shutdown_connections 2024-05-03T20:34:05.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:05.904+0000 7f931ce7d700 1 -- 172.21.15.181:0/3765512938 wait complete. 2024-05-03T20:34:06.563 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-03T20:34:06.803 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:34:07.824 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:07 smithi181 ceph-mon[29801]: pgmap v297: 97 pgs: 97 active+clean; 585 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.2 KiB/s wr, 20 op/s 2024-05-03T20:34:07.824 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:07 smithi181 ceph-mon[29801]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:34:07.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:07 smithi196 ceph-mon[32858]: pgmap v297: 97 pgs: 97 active+clean; 585 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.2 KiB/s wr, 20 op/s 2024-05-03T20:34:07.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:07 smithi196 ceph-mon[32858]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:34:07.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:07 smithi204 ceph-mon[33595]: pgmap v297: 97 pgs: 97 active+clean; 585 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.2 KiB/s wr, 20 op/s 2024-05-03T20:34:07.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:07 smithi204 ceph-mon[33595]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:34:08.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.326+0000 7f3c6979d700 1 -- 172.21.15.181:0/3172878707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c64102450 msgr2=0x7f3c64110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:08.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.326+0000 7f3c6979d700 1 --2- 172.21.15.181:0/3172878707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c64102450 0x7f3c64110340 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00b0a0 tx=0x7f3c5c00b3b0 comp rx=0 tx=0).stop 2024-05-03T20:34:08.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.326+0000 7f3c6979d700 1 -- 172.21.15.181:0/3172878707 shutdown_connections 2024-05-03T20:34:08.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.326+0000 7f3c6979d700 1 --2- 172.21.15.181:0/3172878707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c64102450 0x7f3c64110340 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:08.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.326+0000 7f3c6979d700 1 --2- 172.21.15.181:0/3172878707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c64109520 0x7f3c64109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:08.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.326+0000 7f3c6979d700 1 --2- 172.21.15.181:0/3172878707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c64101a90 0x7f3c64101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:08.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.326+0000 7f3c6979d700 1 -- 172.21.15.181:0/3172878707 >> 172.21.15.181:0/3172878707 conn(0x7f3c64078a10 msgr2=0x7f3c640ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:08.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.326+0000 7f3c6979d700 1 -- 172.21.15.181:0/3172878707 shutdown_connections 2024-05-03T20:34:08.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.326+0000 7f3c6979d700 1 -- 172.21.15.181:0/3172878707 wait complete. 2024-05-03T20:34:08.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.326+0000 7f3c6979d700 1 Processor -- start 2024-05-03T20:34:08.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.327+0000 7f3c6979d700 1 -- start start 2024-05-03T20:34:08.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.327+0000 7f3c6979d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c64101a90 0x7f3c6419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:08.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.327+0000 7f3c6979d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c64102450 0x7f3c6419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:08.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.327+0000 7f3c6979d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c64109520 0x7f3c64199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:08.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.327+0000 7f3c6979d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3c641136d0 con 0x7f3c64109520 2024-05-03T20:34:08.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.327+0000 7f3c6979d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3c64113550 con 0x7f3c64102450 2024-05-03T20:34:08.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.327+0000 7f3c6979d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3c64113850 con 0x7f3c64101a90 2024-05-03T20:34:08.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.327+0000 7f3c637fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c64109520 0x7f3c64199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:08.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.327+0000 7f3c637fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c64109520 0x7f3c64199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54396/0 (socket says 172.21.15.181:54396) 2024-05-03T20:34:08.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.327+0000 7f3c637fe700 1 -- 172.21.15.181:0/2628156604 learned_addr learned my addr 172.21.15.181:0/2628156604 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:34:08.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.327+0000 7f3c62ffd700 1 --2- 172.21.15.181:0/2628156604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c64101a90 0x7f3c6419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:08.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.327+0000 7f3c5a7fc700 1 --2- 172.21.15.181:0/2628156604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c64102450 0x7f3c6419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:08.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.327+0000 7f3c637fe700 1 -- 172.21.15.181:0/2628156604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c64101a90 msgr2=0x7f3c6419f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:08.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.327+0000 7f3c637fe700 1 --2- 172.21.15.181:0/2628156604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c64101a90 0x7f3c6419f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:08.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.327+0000 7f3c637fe700 1 -- 172.21.15.181:0/2628156604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c64102450 msgr2=0x7f3c6419fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:08.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.327+0000 7f3c637fe700 1 --2- 172.21.15.181:0/2628156604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c64102450 0x7f3c6419fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:08.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.327+0000 7f3c637fe700 1 -- 172.21.15.181:0/2628156604 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c6419a0d0 con 0x7f3c64109520 2024-05-03T20:34:08.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.327+0000 7f3c5a7fc700 1 --2- 172.21.15.181:0/2628156604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c64102450 0x7f3c6419fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:34:08.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.327+0000 7f3c637fe700 1 --2- 172.21.15.181:0/2628156604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c64109520 0x7f3c64199970 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00b830 tx=0x7f3c5c004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:08.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.328+0000 7f3c60ff9700 1 -- 172.21.15.181:0/2628156604 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c021070 con 0x7f3c64109520 2024-05-03T20:34:08.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.328+0000 7f3c6979d700 1 -- 172.21.15.181:0/2628156604 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c6419a360 con 0x7f3c64109520 2024-05-03T20:34:08.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.328+0000 7f3c60ff9700 1 -- 172.21.15.181:0/2628156604 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c5c007e30 con 0x7f3c64109520 2024-05-03T20:34:08.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.328+0000 7f3c60ff9700 1 -- 172.21.15.181:0/2628156604 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c0047d0 con 0x7f3c64109520 2024-05-03T20:34:08.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.328+0000 7f3c6979d700 1 -- 172.21.15.181:0/2628156604 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c641a6680 con 0x7f3c64109520 2024-05-03T20:34:08.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.328+0000 7f3c6979d700 1 -- 172.21.15.181:0/2628156604 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c6410ba00 con 0x7f3c64109520 2024-05-03T20:34:08.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.330+0000 7f3c60ff9700 1 -- 172.21.15.181:0/2628156604 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f3c5c007980 con 0x7f3c64109520 2024-05-03T20:34:08.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.330+0000 7f3c60ff9700 1 --2- 172.21.15.181:0/2628156604 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3c500b2910 0x7f3c500b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:08.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.330+0000 7f3c62ffd700 1 --2- 172.21.15.181:0/2628156604 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3c500b2910 0x7f3c500b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:08.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.330+0000 7f3c60ff9700 1 -- 172.21.15.181:0/2628156604 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f3c5c00f020 con 0x7f3c64109520 2024-05-03T20:34:08.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.331+0000 7f3c62ffd700 1 --2- 172.21.15.181:0/2628156604 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3c500b2910 0x7f3c500b4dd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3c6419aab0 tx=0x7f3c5400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:08.332 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.334+0000 7f3c60ff9700 1 -- 172.21.15.181:0/2628156604 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3c5c02a020 con 0x7f3c64109520 2024-05-03T20:34:08.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.580+0000 7f3c6979d700 1 -- 172.21.15.181:0/2628156604 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f3c640638e0 con 0x7f3c500b2910 2024-05-03T20:34:08.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.585+0000 7f3c60ff9700 1 -- 172.21.15.181:0/2628156604 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f3c640638e0 con 0x7f3c500b2910 2024-05-03T20:34:08.584 INFO:teuthology.orchestra.run.smithi181.stdout:/volumes/qa/sv_0/353db3fe-1104-4f2c-af5b-96436c9afd52 2024-05-03T20:34:08.586 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.587+0000 7f3c6979d700 1 -- 172.21.15.181:0/2628156604 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3c500b2910 msgr2=0x7f3c500b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:08.586 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.587+0000 7f3c6979d700 1 --2- 172.21.15.181:0/2628156604 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3c500b2910 0x7f3c500b4dd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3c6419aab0 tx=0x7f3c5400b3c0 comp rx=0 tx=0).stop 2024-05-03T20:34:08.586 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.587+0000 7f3c6979d700 1 -- 172.21.15.181:0/2628156604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c64109520 msgr2=0x7f3c64199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:08.586 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.587+0000 7f3c6979d700 1 --2- 172.21.15.181:0/2628156604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c64109520 0x7f3c64199970 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00b830 tx=0x7f3c5c004ce0 comp rx=0 tx=0).stop 2024-05-03T20:34:08.586 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.587+0000 7f3c6979d700 1 -- 172.21.15.181:0/2628156604 shutdown_connections 2024-05-03T20:34:08.586 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.587+0000 7f3c6979d700 1 --2- 172.21.15.181:0/2628156604 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3c500b2910 0x7f3c500b4dd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:08.586 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.587+0000 7f3c6979d700 1 --2- 172.21.15.181:0/2628156604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c64109520 0x7f3c64199970 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:08.587 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.587+0000 7f3c6979d700 1 --2- 172.21.15.181:0/2628156604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c64101a90 0x7f3c6419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:08.587 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.587+0000 7f3c6979d700 1 --2- 172.21.15.181:0/2628156604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c64102450 0x7f3c6419fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:08.587 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.587+0000 7f3c6979d700 1 -- 172.21.15.181:0/2628156604 >> 172.21.15.181:0/2628156604 conn(0x7f3c64078a10 msgr2=0x7f3c64100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:08.587 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.588+0000 7f3c6979d700 1 -- 172.21.15.181:0/2628156604 shutdown_connections 2024-05-03T20:34:08.587 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:08.588+0000 7f3c6979d700 1 -- 172.21.15.181:0/2628156604 wait complete. 2024-05-03T20:34:09.414 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-03T20:34:09.659 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:34:09.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:09 smithi181 ceph-mon[29801]: pgmap v298: 97 pgs: 97 active+clean; 586 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.2 KiB/s wr, 20 op/s 2024-05-03T20:34:09.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:09 smithi181 ceph-mon[29801]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:34:09.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:09 smithi196 ceph-mon[32858]: pgmap v298: 97 pgs: 97 active+clean; 586 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.2 KiB/s wr, 20 op/s 2024-05-03T20:34:09.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:09 smithi196 ceph-mon[32858]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:34:09.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:09 smithi204 ceph-mon[33595]: pgmap v298: 97 pgs: 97 active+clean; 586 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.2 KiB/s wr, 20 op/s 2024-05-03T20:34:09.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:09 smithi204 ceph-mon[33595]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:34:11.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.234+0000 7f869235e700 1 -- 172.21.15.181:0/1986583232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f868c108410 msgr2=0x7f868c1087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:11.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.234+0000 7f869235e700 1 --2- 172.21.15.181:0/1986583232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f868c108410 0x7f868c1087f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f867c009990 tx=0x7f867c009ca0 comp rx=0 tx=0).stop 2024-05-03T20:34:11.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.234+0000 7f869235e700 1 -- 172.21.15.181:0/1986583232 shutdown_connections 2024-05-03T20:34:11.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.234+0000 7f869235e700 1 --2- 172.21.15.181:0/1986583232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f868c102d60 0x7f868c10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:11.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.234+0000 7f869235e700 1 --2- 172.21.15.181:0/1986583232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f868c108410 0x7f868c1087f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:11.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.234+0000 7f869235e700 1 --2- 172.21.15.181:0/1986583232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868c1023a0 0x7f868c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:11.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.234+0000 7f869235e700 1 -- 172.21.15.181:0/1986583232 >> 172.21.15.181:0/1986583232 conn(0x7f868c079350 msgr2=0x7f868c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:11.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.234+0000 7f869235e700 1 -- 172.21.15.181:0/1986583232 shutdown_connections 2024-05-03T20:34:11.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.234+0000 7f869235e700 1 -- 172.21.15.181:0/1986583232 wait complete. 2024-05-03T20:34:11.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.235+0000 7f869235e700 1 Processor -- start 2024-05-03T20:34:11.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.235+0000 7f869235e700 1 -- start start 2024-05-03T20:34:11.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.235+0000 7f869235e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868c1023a0 0x7f868c19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:11.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.235+0000 7f869235e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f868c102d60 0x7f868c19afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:11.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.235+0000 7f869235e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f868c108410 0x7f868c1a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:11.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.235+0000 7f869235e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f868c112560 con 0x7f868c108410 2024-05-03T20:34:11.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.235+0000 7f869235e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f868c1123e0 con 0x7f868c102d60 2024-05-03T20:34:11.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.235+0000 7f869235e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f868c1126e0 con 0x7f868c1023a0 2024-05-03T20:34:11.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.235+0000 7f86908fb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f868c108410 0x7f868c1a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:11.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.235+0000 7f86908fb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f868c108410 0x7f868c1a5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54422/0 (socket says 172.21.15.181:54422) 2024-05-03T20:34:11.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.235+0000 7f86908fb700 1 -- 172.21.15.181:0/3278256675 learned_addr learned my addr 172.21.15.181:0/3278256675 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:34:11.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.236+0000 7f868b7fe700 1 --2- 172.21.15.181:0/3278256675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f868c102d60 0x7f868c19afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:11.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.236+0000 7f868bfff700 1 --2- 172.21.15.181:0/3278256675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868c1023a0 0x7f868c19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:11.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.236+0000 7f86908fb700 1 -- 172.21.15.181:0/3278256675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868c1023a0 msgr2=0x7f868c19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:11.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.236+0000 7f86908fb700 1 --2- 172.21.15.181:0/3278256675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868c1023a0 0x7f868c19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:11.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.236+0000 7f86908fb700 1 -- 172.21.15.181:0/3278256675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f868c102d60 msgr2=0x7f868c19afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:11.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.236+0000 7f86908fb700 1 --2- 172.21.15.181:0/3278256675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f868c102d60 0x7f868c19afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:11.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.236+0000 7f86908fb700 1 -- 172.21.15.181:0/3278256675 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868c1a6020 con 0x7f868c108410 2024-05-03T20:34:11.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.236+0000 7f86908fb700 1 --2- 172.21.15.181:0/3278256675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f868c108410 0x7f868c1a5ae0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8684007ee0 tx=0x7f868400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:11.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.236+0000 7f86897fa700 1 -- 172.21.15.181:0/3278256675 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684004bb0 con 0x7f868c108410 2024-05-03T20:34:11.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.236+0000 7f86897fa700 1 -- 172.21.15.181:0/3278256675 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8684004d50 con 0x7f868c108410 2024-05-03T20:34:11.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.236+0000 7f86897fa700 1 -- 172.21.15.181:0/3278256675 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684005920 con 0x7f868c108410 2024-05-03T20:34:11.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.236+0000 7f869235e700 1 -- 172.21.15.181:0/3278256675 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f868c1a6250 con 0x7f868c108410 2024-05-03T20:34:11.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.236+0000 7f869235e700 1 -- 172.21.15.181:0/3278256675 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f868c1a6760 con 0x7f868c108410 2024-05-03T20:34:11.238 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.237+0000 7f869235e700 1 -- 172.21.15.181:0/3278256675 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f868c06a190 con 0x7f868c108410 2024-05-03T20:34:11.238 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.238+0000 7f86897fa700 1 -- 172.21.15.181:0/3278256675 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f868401a020 con 0x7f868c108410 2024-05-03T20:34:11.238 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.239+0000 7f86897fa700 1 --2- 172.21.15.181:0/3278256675 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f86740b2990 0x7f86740b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:11.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.239+0000 7f868bfff700 1 --2- 172.21.15.181:0/3278256675 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f86740b2990 0x7f86740b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:11.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.239+0000 7f86897fa700 1 -- 172.21.15.181:0/3278256675 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f86840e1780 con 0x7f868c108410 2024-05-03T20:34:11.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.240+0000 7f868bfff700 1 --2- 172.21.15.181:0/3278256675 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f86740b2990 0x7f86740b4e50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f867c00b5c0 tx=0x7f867c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:11.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.244+0000 7f86897fa700 1 -- 172.21.15.181:0/3278256675 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8684010070 con 0x7f868c108410 2024-05-03T20:34:11.477 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.478+0000 7f869235e700 1 -- 172.21.15.181:0/3278256675 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f868c0638e0 con 0x7f86740b2990 2024-05-03T20:34:11.507 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.508+0000 7f86897fa700 1 -- 172.21.15.181:0/3278256675 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f868c0638e0 con 0x7f86740b2990 2024-05-03T20:34:11.509 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.511+0000 7f869235e700 1 -- 172.21.15.181:0/3278256675 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f86740b2990 msgr2=0x7f86740b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:11.510 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.511+0000 7f869235e700 1 --2- 172.21.15.181:0/3278256675 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f86740b2990 0x7f86740b4e50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f867c00b5c0 tx=0x7f867c005960 comp rx=0 tx=0).stop 2024-05-03T20:34:11.510 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.511+0000 7f869235e700 1 -- 172.21.15.181:0/3278256675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f868c108410 msgr2=0x7f868c1a5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:11.510 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.511+0000 7f869235e700 1 --2- 172.21.15.181:0/3278256675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f868c108410 0x7f868c1a5ae0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8684007ee0 tx=0x7f868400d3b0 comp rx=0 tx=0).stop 2024-05-03T20:34:11.510 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.511+0000 7f869235e700 1 -- 172.21.15.181:0/3278256675 shutdown_connections 2024-05-03T20:34:11.510 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.511+0000 7f869235e700 1 --2- 172.21.15.181:0/3278256675 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f86740b2990 0x7f86740b4e50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:11.510 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.511+0000 7f869235e700 1 --2- 172.21.15.181:0/3278256675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f868c108410 0x7f868c1a5ae0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:11.510 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.511+0000 7f869235e700 1 --2- 172.21.15.181:0/3278256675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868c1023a0 0x7f868c19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:11.510 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.511+0000 7f869235e700 1 --2- 172.21.15.181:0/3278256675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f868c102d60 0x7f868c19afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:11.510 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.511+0000 7f869235e700 1 -- 172.21.15.181:0/3278256675 >> 172.21.15.181:0/3278256675 conn(0x7f868c079350 msgr2=0x7f868c101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:11.510 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.511+0000 7f869235e700 1 -- 172.21.15.181:0/3278256675 shutdown_connections 2024-05-03T20:34:11.510 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:11.511+0000 7f869235e700 1 -- 172.21.15.181:0/3278256675 wait complete. 2024-05-03T20:34:11.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:11 smithi196 ceph-mon[32858]: pgmap v299: 97 pgs: 97 active+clean; 606 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.4 KiB/s wr, 19 op/s 2024-05-03T20:34:11.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:11 smithi204 ceph-mon[33595]: pgmap v299: 97 pgs: 97 active+clean; 606 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.4 KiB/s wr, 19 op/s 2024-05-03T20:34:11.901 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:11 smithi181 ceph-mon[29801]: pgmap v299: 97 pgs: 97 active+clean; 606 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.4 KiB/s wr, 19 op/s 2024-05-03T20:34:12.166 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-03T20:34:12.416 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:34:12.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:12 smithi181 ceph-mon[29801]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:34:12.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:12 smithi196 ceph-mon[32858]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:34:12.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:12 smithi204 ceph-mon[33595]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:34:13.853 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:13 smithi181 ceph-mon[29801]: pgmap v300: 97 pgs: 97 active+clean; 606 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.3 KiB/s wr, 19 op/s 2024-05-03T20:34:13.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:13 smithi196 ceph-mon[32858]: pgmap v300: 97 pgs: 97 active+clean; 606 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.3 KiB/s wr, 19 op/s 2024-05-03T20:34:13.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:13 smithi204 ceph-mon[33595]: pgmap v300: 97 pgs: 97 active+clean; 606 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.3 KiB/s wr, 19 op/s 2024-05-03T20:34:14.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.091+0000 7fb313674700 1 -- 172.21.15.181:0/2159144095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb30c105ed0 msgr2=0x7fb30c106380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:14.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.091+0000 7fb313674700 1 --2- 172.21.15.181:0/2159144095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb30c105ed0 0x7fb30c106380 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb30400b0a0 tx=0x7fb30400b3b0 comp rx=0 tx=0).stop 2024-05-03T20:34:14.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.091+0000 7fb313674700 1 -- 172.21.15.181:0/2159144095 shutdown_connections 2024-05-03T20:34:14.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.091+0000 7fb313674700 1 --2- 172.21.15.181:0/2159144095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb30c105ed0 0x7fb30c106380 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:14.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.091+0000 7fb313674700 1 --2- 172.21.15.181:0/2159144095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb30c1083c0 0x7fb30c06bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:14.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.091+0000 7fb313674700 1 --2- 172.21.15.181:0/2159144095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb30c06c130 0x7fb30c06c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:14.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.091+0000 7fb313674700 1 -- 172.21.15.181:0/2159144095 >> 172.21.15.181:0/2159144095 conn(0x7fb30c079370 msgr2=0x7fb30c079780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:14.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.091+0000 7fb313674700 1 -- 172.21.15.181:0/2159144095 shutdown_connections 2024-05-03T20:34:14.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.091+0000 7fb313674700 1 -- 172.21.15.181:0/2159144095 wait complete. 2024-05-03T20:34:14.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.092+0000 7fb313674700 1 Processor -- start 2024-05-03T20:34:14.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.092+0000 7fb313674700 1 -- start start 2024-05-03T20:34:14.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.092+0000 7fb313674700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb30c06c130 0x7fb30c1a02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:14.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.092+0000 7fb313674700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb30c105ed0 0x7fb30c1a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:14.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.092+0000 7fb313674700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb30c1083c0 0x7fb30c1a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:14.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.092+0000 7fb313674700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c119d20 con 0x7fb30c06c130 2024-05-03T20:34:14.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.092+0000 7fb313674700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c119ba0 con 0x7fb30c1083c0 2024-05-03T20:34:14.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.092+0000 7fb313674700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c119ea0 con 0x7fb30c105ed0 2024-05-03T20:34:14.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.092+0000 7fb311410700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb30c06c130 0x7fb30c1a02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:14.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.092+0000 7fb311410700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb30c06c130 0x7fb30c1a02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54440/0 (socket says 172.21.15.181:54440) 2024-05-03T20:34:14.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.092+0000 7fb311410700 1 -- 172.21.15.181:0/1012205320 learned_addr learned my addr 172.21.15.181:0/1012205320 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:34:14.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.092+0000 7fb310c0f700 1 --2- 172.21.15.181:0/1012205320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb30c105ed0 0x7fb30c1a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:14.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.092+0000 7fb311c11700 1 --2- 172.21.15.181:0/1012205320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb30c1083c0 0x7fb30c1a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:14.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.092+0000 7fb311410700 1 -- 172.21.15.181:0/1012205320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb30c105ed0 msgr2=0x7fb30c1a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:14.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.092+0000 7fb311410700 1 --2- 172.21.15.181:0/1012205320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb30c105ed0 0x7fb30c1a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:14.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.092+0000 7fb311410700 1 -- 172.21.15.181:0/1012205320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb30c1083c0 msgr2=0x7fb30c1a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:14.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.092+0000 7fb311410700 1 --2- 172.21.15.181:0/1012205320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb30c1083c0 0x7fb30c1a4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:14.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.092+0000 7fb311410700 1 -- 172.21.15.181:0/1012205320 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30c1a52e0 con 0x7fb30c06c130 2024-05-03T20:34:14.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.092+0000 7fb310c0f700 1 --2- 172.21.15.181:0/1012205320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb30c105ed0 0x7fb30c1a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:34:14.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.093+0000 7fb311410700 1 --2- 172.21.15.181:0/1012205320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb30c06c130 0x7fb30c1a02d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb30800b530 tx=0x7fb30800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:14.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.093+0000 7fb3027fc700 1 -- 172.21.15.181:0/1012205320 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb308015020 con 0x7fb30c06c130 2024-05-03T20:34:14.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.093+0000 7fb3027fc700 1 -- 172.21.15.181:0/1012205320 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb308011ce0 con 0x7fb30c06c130 2024-05-03T20:34:14.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.093+0000 7fb313674700 1 -- 172.21.15.181:0/1012205320 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb30c1a55d0 con 0x7fb30c06c130 2024-05-03T20:34:14.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.093+0000 7fb3027fc700 1 -- 172.21.15.181:0/1012205320 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb308010d20 con 0x7fb30c06c130 2024-05-03T20:34:14.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.093+0000 7fb313674700 1 -- 172.21.15.181:0/1012205320 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb30c1ace50 con 0x7fb30c06c130 2024-05-03T20:34:14.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.094+0000 7fb313674700 1 -- 172.21.15.181:0/1012205320 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb30c06a190 con 0x7fb30c06c130 2024-05-03T20:34:14.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.095+0000 7fb3027fc700 1 -- 172.21.15.181:0/1012205320 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fb308018020 con 0x7fb30c06c130 2024-05-03T20:34:14.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.095+0000 7fb3027fc700 1 --2- 172.21.15.181:0/1012205320 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb2f80b2990 0x7fb2f80b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:14.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.096+0000 7fb310c0f700 1 --2- 172.21.15.181:0/1012205320 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb2f80b2990 0x7fb2f80b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:14.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.096+0000 7fb3027fc700 1 -- 172.21.15.181:0/1012205320 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fb3080e1e00 con 0x7fb30c06c130 2024-05-03T20:34:14.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.096+0000 7fb310c0f700 1 --2- 172.21.15.181:0/1012205320 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb2f80b2990 0x7fb2f80b4e50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb30c079b10 tx=0x7fb2fc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:14.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.099+0000 7fb3027fc700 1 -- 172.21.15.181:0/1012205320 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb30801f020 con 0x7fb30c06c130 2024-05-03T20:34:14.336 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.338+0000 7fb313674700 1 -- 172.21.15.181:0/1012205320 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fb30c1055c0 con 0x7fb2f80b2990 2024-05-03T20:34:14.341 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.343+0000 7fb3027fc700 1 -- 172.21.15.181:0/1012205320 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fb30c1055c0 con 0x7fb2f80b2990 2024-05-03T20:34:14.342 INFO:teuthology.orchestra.run.smithi181.stdout:/volumes/qa/sv_1/541f494e-3959-4449-bf1a-b4e2dce12c39 2024-05-03T20:34:14.344 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.345+0000 7fb313674700 1 -- 172.21.15.181:0/1012205320 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb2f80b2990 msgr2=0x7fb2f80b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:14.344 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.345+0000 7fb313674700 1 --2- 172.21.15.181:0/1012205320 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb2f80b2990 0x7fb2f80b4e50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb30c079b10 tx=0x7fb2fc008040 comp rx=0 tx=0).stop 2024-05-03T20:34:14.344 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.345+0000 7fb313674700 1 -- 172.21.15.181:0/1012205320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb30c06c130 msgr2=0x7fb30c1a02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:14.344 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.345+0000 7fb313674700 1 --2- 172.21.15.181:0/1012205320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb30c06c130 0x7fb30c1a02d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb30800b530 tx=0x7fb30800b8f0 comp rx=0 tx=0).stop 2024-05-03T20:34:14.344 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.345+0000 7fb313674700 1 -- 172.21.15.181:0/1012205320 shutdown_connections 2024-05-03T20:34:14.344 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.345+0000 7fb313674700 1 --2- 172.21.15.181:0/1012205320 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb2f80b2990 0x7fb2f80b4e50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:14.344 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.345+0000 7fb313674700 1 --2- 172.21.15.181:0/1012205320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb30c1083c0 0x7fb30c1a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:14.344 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.345+0000 7fb313674700 1 --2- 172.21.15.181:0/1012205320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb30c06c130 0x7fb30c1a02d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:14.344 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.345+0000 7fb313674700 1 --2- 172.21.15.181:0/1012205320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb30c105ed0 0x7fb30c1a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:14.344 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.345+0000 7fb313674700 1 -- 172.21.15.181:0/1012205320 >> 172.21.15.181:0/1012205320 conn(0x7fb30c079370 msgr2=0x7fb30c104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:14.344 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.345+0000 7fb313674700 1 -- 172.21.15.181:0/1012205320 shutdown_connections 2024-05-03T20:34:14.344 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:14.345+0000 7fb313674700 1 -- 172.21.15.181:0/1012205320 wait complete. 2024-05-03T20:34:14.949 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T20:34:15.188 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:34:15.625 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:15 smithi181 ceph-mon[29801]: pgmap v301: 97 pgs: 97 active+clean; 624 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-05-03T20:34:15.625 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:15 smithi181 ceph-mon[29801]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:34:15.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:15 smithi196 ceph-mon[32858]: pgmap v301: 97 pgs: 97 active+clean; 624 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-05-03T20:34:15.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:15 smithi196 ceph-mon[32858]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:34:15.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:15 smithi204 ceph-mon[33595]: pgmap v301: 97 pgs: 97 active+clean; 624 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-05-03T20:34:15.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:15 smithi204 ceph-mon[33595]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:34:16.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.867+0000 7fd62238c700 1 -- 172.21.15.181:0/2482862446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd61c104100 msgr2=0x7fd61c108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:16.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.867+0000 7fd62238c700 1 --2- 172.21.15.181:0/2482862446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd61c104100 0x7fd61c108120 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd60c0099c0 tx=0x7fd60c009cd0 comp rx=0 tx=0).stop 2024-05-03T20:34:16.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.867+0000 7fd62238c700 1 -- 172.21.15.181:0/2482862446 shutdown_connections 2024-05-03T20:34:16.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.867+0000 7fd62238c700 1 --2- 172.21.15.181:0/2482862446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd61c108850 0x7fd61c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:16.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.867+0000 7fd62238c700 1 --2- 172.21.15.181:0/2482862446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd61c103750 0x7fd61c103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:16.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.867+0000 7fd62238c700 1 --2- 172.21.15.181:0/2482862446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd61c104100 0x7fd61c108120 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:16.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.867+0000 7fd62238c700 1 -- 172.21.15.181:0/2482862446 >> 172.21.15.181:0/2482862446 conn(0x7fd61c07a600 msgr2=0x7fd61c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:16.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.867+0000 7fd62238c700 1 -- 172.21.15.181:0/2482862446 shutdown_connections 2024-05-03T20:34:16.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.867+0000 7fd62238c700 1 -- 172.21.15.181:0/2482862446 wait complete. 2024-05-03T20:34:16.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.868+0000 7fd62238c700 1 Processor -- start 2024-05-03T20:34:16.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.868+0000 7fd62238c700 1 -- start start 2024-05-03T20:34:16.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.868+0000 7fd62238c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd61c103750 0x7fd61c0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:16.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.868+0000 7fd62238c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd61c104100 0x7fd61c075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:16.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.868+0000 7fd62238c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd61c108850 0x7fd61c076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:16.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.868+0000 7fd62238c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c113690 con 0x7fd61c104100 2024-05-03T20:34:16.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.868+0000 7fd62238c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c113510 con 0x7fd61c103750 2024-05-03T20:34:16.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.868+0000 7fd62238c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c113810 con 0x7fd61c108850 2024-05-03T20:34:16.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.869+0000 7fd61b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd61c104100 0x7fd61c075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:16.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.869+0000 7fd61b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd61c104100 0x7fd61c075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48952/0 (socket says 172.21.15.181:48952) 2024-05-03T20:34:16.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.869+0000 7fd61b7fe700 1 -- 172.21.15.181:0/1926945198 learned_addr learned my addr 172.21.15.181:0/1926945198 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:34:16.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.869+0000 7fd61bfff700 1 --2- 172.21.15.181:0/1926945198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd61c103750 0x7fd61c0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:16.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.869+0000 7fd61b7fe700 1 -- 172.21.15.181:0/1926945198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd61c108850 msgr2=0x7fd61c076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:16.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.869+0000 7fd620929700 1 --2- 172.21.15.181:0/1926945198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd61c108850 0x7fd61c076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:16.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.869+0000 7fd61b7fe700 1 --2- 172.21.15.181:0/1926945198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd61c108850 0x7fd61c076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:16.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.869+0000 7fd61b7fe700 1 -- 172.21.15.181:0/1926945198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd61c103750 msgr2=0x7fd61c0757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:16.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.869+0000 7fd61b7fe700 1 --2- 172.21.15.181:0/1926945198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd61c103750 0x7fd61c0757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:16.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.869+0000 7fd61b7fe700 1 -- 172.21.15.181:0/1926945198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61c1a8070 con 0x7fd61c104100 2024-05-03T20:34:16.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.869+0000 7fd61b7fe700 1 --2- 172.21.15.181:0/1926945198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd61c104100 0x7fd61c075ce0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd60c0098e0 tx=0x7fd60c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:16.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.869+0000 7fd6197fa700 1 -- 172.21.15.181:0/1926945198 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c027070 con 0x7fd61c104100 2024-05-03T20:34:16.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.869+0000 7fd62238c700 1 -- 172.21.15.181:0/1926945198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd61c1a8300 con 0x7fd61c104100 2024-05-03T20:34:16.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.869+0000 7fd6197fa700 1 -- 172.21.15.181:0/1926945198 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd60c003f20 con 0x7fd61c104100 2024-05-03T20:34:16.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.869+0000 7fd6197fa700 1 -- 172.21.15.181:0/1926945198 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c022da0 con 0x7fd61c104100 2024-05-03T20:34:16.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.869+0000 7fd62238c700 1 -- 172.21.15.181:0/1926945198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd61c1a87b0 con 0x7fd61c104100 2024-05-03T20:34:16.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.870+0000 7fd62238c700 1 -- 172.21.15.181:0/1926945198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd61c06a190 con 0x7fd61c104100 2024-05-03T20:34:16.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.872+0000 7fd6197fa700 1 -- 172.21.15.181:0/1926945198 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fd60c02a070 con 0x7fd61c104100 2024-05-03T20:34:16.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.872+0000 7fd6197fa700 1 --2- 172.21.15.181:0/1926945198 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd6040b2910 0x7fd6040b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:16.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.872+0000 7fd61bfff700 1 --2- 172.21.15.181:0/1926945198 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd6040b2910 0x7fd6040b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:16.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.872+0000 7fd6197fa700 1 -- 172.21.15.181:0/1926945198 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fd60c0e36a0 con 0x7fd61c104100 2024-05-03T20:34:16.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.873+0000 7fd61bfff700 1 --2- 172.21.15.181:0/1926945198 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd6040b2910 0x7fd6040b4dd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd610009a10 tx=0x7fd610009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:16.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:16.876+0000 7fd6197fa700 1 -- 172.21.15.181:0/1926945198 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd60c031020 con 0x7fd61c104100 2024-05-03T20:34:17.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:17.125+0000 7fd62238c700 1 -- 172.21.15.181:0/1926945198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd61c02e510 con 0x7fd61c104100 2024-05-03T20:34:17.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:17.125+0000 7fd6197fa700 1 -- 172.21.15.181:0/1926945198 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11954 (secure 0 0 0) 0x7fd60c013070 con 0x7fd61c104100 2024-05-03T20:34:17.124 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:34:17.125 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2414172454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2414172454},{"type":"v1","addr":"172.21.15.181:6835","nonce":2414172454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/204127189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":204127189},{"type":"v1","addr":"172.21.15.181:6837","nonce":204127189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T20:33:42.487965+0000","modified":"2024-05-03T20:34:00.359735+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24475,"mds_2":24464,"mds_3":24463,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14562":{"gid":14562,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6839/257361276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":257361276},{"type":"v1","addr":"172.21.15.181:6839","nonce":257361276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6841/2403160115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2403160115},{"type":"v1","addr":"172.21.15.181:6841","nonce":2403160115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3975688022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3975688022},{"type":"v1","addr":"172.21.15.196:6833","nonce":3975688022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2409628823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2409628823},{"type":"v1","addr":"172.21.15.204:6833","nonce":2409628823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1820206106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1820206106},{"type":"v1","addr":"172.21.15.196:6835","nonce":1820206106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6835/3870850287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3870850287},{"type":"v1","addr":"172.21.15.204:6835","nonce":3870850287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/3006728559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3006728559},{"type":"v1","addr":"172.21.15.204:6837","nonce":3006728559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6837/1529420259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1529420259},{"type":"v1","addr":"172.21.15.196:6837","nonce":1529420259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/1644828559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1644828559},{"type":"v1","addr":"172.21.15.204:6839","nonce":1644828559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.196:6839/1182826743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1182826743},{"type":"v1","addr":"172.21.15.196:6839","nonce":1182826743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24457,24464,24463,24475,24488]},"id":1}]} 2024-05-03T20:34:17.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:17.128+0000 7fd62238c700 1 -- 172.21.15.181:0/1926945198 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd6040b2910 msgr2=0x7fd6040b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:17.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:17.128+0000 7fd62238c700 1 --2- 172.21.15.181:0/1926945198 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd6040b2910 0x7fd6040b4dd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd610009a10 tx=0x7fd610009190 comp rx=0 tx=0).stop 2024-05-03T20:34:17.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:17.128+0000 7fd62238c700 1 -- 172.21.15.181:0/1926945198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd61c104100 msgr2=0x7fd61c075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:17.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:17.128+0000 7fd62238c700 1 --2- 172.21.15.181:0/1926945198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd61c104100 0x7fd61c075ce0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd60c0098e0 tx=0x7fd60c0057c0 comp rx=0 tx=0).stop 2024-05-03T20:34:17.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:17.128+0000 7fd62238c700 1 -- 172.21.15.181:0/1926945198 shutdown_connections 2024-05-03T20:34:17.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:17.128+0000 7fd62238c700 1 --2- 172.21.15.181:0/1926945198 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd6040b2910 0x7fd6040b4dd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:17.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:17.128+0000 7fd62238c700 1 --2- 172.21.15.181:0/1926945198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd61c108850 0x7fd61c076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:17.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:17.128+0000 7fd62238c700 1 --2- 172.21.15.181:0/1926945198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd61c103750 0x7fd61c0757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:17.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:17.128+0000 7fd62238c700 1 --2- 172.21.15.181:0/1926945198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd61c104100 0x7fd61c075ce0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:17.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:17.128+0000 7fd62238c700 1 -- 172.21.15.181:0/1926945198 >> 172.21.15.181:0/1926945198 conn(0x7fd61c07a600 msgr2=0x7fd61c101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:17.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:17.128+0000 7fd62238c700 1 -- 172.21.15.181:0/1926945198 shutdown_connections 2024-05-03T20:34:17.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:17.128+0000 7fd62238c700 1 -- 172.21.15.181:0/1926945198 wait complete. 2024-05-03T20:34:17.128 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 27 2024-05-03T20:34:17.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:17 smithi181 ceph-mon[29801]: pgmap v302: 97 pgs: 97 active+clean; 624 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.3 KiB/s wr, 12 op/s 2024-05-03T20:34:17.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:17 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/1926945198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:34:17.845 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T20:34:17.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:17 smithi196 ceph-mon[32858]: pgmap v302: 97 pgs: 97 active+clean; 624 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.3 KiB/s wr, 12 op/s 2024-05-03T20:34:17.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:17 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/1926945198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:34:17.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:17 smithi204 ceph-mon[33595]: pgmap v302: 97 pgs: 97 active+clean; 624 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.3 KiB/s wr, 12 op/s 2024-05-03T20:34:17.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:17 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/1926945198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:34:18.088 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:34:19.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.750+0000 7fb4daf1b700 1 -- 172.21.15.181:0/1061587017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4d4108410 msgr2=0x7fb4d41087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:19.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.750+0000 7fb4daf1b700 1 --2- 172.21.15.181:0/1061587017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4d4108410 0x7fb4d41087f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8009990 tx=0x7fb4c8009ca0 comp rx=0 tx=0).stop 2024-05-03T20:34:19.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.750+0000 7fb4daf1b700 1 -- 172.21.15.181:0/1061587017 shutdown_connections 2024-05-03T20:34:19.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.750+0000 7fb4daf1b700 1 --2- 172.21.15.181:0/1061587017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4d4102d60 0x7fb4d410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:19.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.750+0000 7fb4daf1b700 1 --2- 172.21.15.181:0/1061587017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4d4108410 0x7fb4d41087f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:19.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.750+0000 7fb4daf1b700 1 --2- 172.21.15.181:0/1061587017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d41023a0 0x7fb4d4102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:19.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.750+0000 7fb4daf1b700 1 -- 172.21.15.181:0/1061587017 >> 172.21.15.181:0/1061587017 conn(0x7fb4d4079350 msgr2=0x7fb4d4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:19.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.750+0000 7fb4daf1b700 1 -- 172.21.15.181:0/1061587017 shutdown_connections 2024-05-03T20:34:19.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.750+0000 7fb4daf1b700 1 -- 172.21.15.181:0/1061587017 wait complete. 2024-05-03T20:34:19.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.751+0000 7fb4daf1b700 1 Processor -- start 2024-05-03T20:34:19.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.751+0000 7fb4daf1b700 1 -- start start 2024-05-03T20:34:19.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.751+0000 7fb4daf1b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4d41023a0 0x7fb4d4104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:19.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.751+0000 7fb4daf1b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4d4102d60 0x7fb4d41045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:19.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.751+0000 7fb4daf1b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d4108410 0x7fb4d4104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:19.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.751+0000 7fb4daf1b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d4112530 con 0x7fb4d4102d60 2024-05-03T20:34:19.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.751+0000 7fb4daf1b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d41123b0 con 0x7fb4d41023a0 2024-05-03T20:34:19.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.751+0000 7fb4daf1b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d41126b0 con 0x7fb4d4108410 2024-05-03T20:34:19.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.751+0000 7fb4d3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4d4102d60 0x7fb4d41045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:19.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.751+0000 7fb4d3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4d4102d60 0x7fb4d41045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48970/0 (socket says 172.21.15.181:48970) 2024-05-03T20:34:19.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.751+0000 7fb4d3fff700 1 -- 172.21.15.181:0/467939527 learned_addr learned my addr 172.21.15.181:0/467939527 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:34:19.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.751+0000 7fb4d8cb7700 1 --2- 172.21.15.181:0/467939527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4d41023a0 0x7fb4d4104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:19.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.751+0000 7fb4d94b8700 1 --2- 172.21.15.181:0/467939527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d4108410 0x7fb4d4104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:19.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.751+0000 7fb4d3fff700 1 -- 172.21.15.181:0/467939527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d4108410 msgr2=0x7fb4d4104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:19.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.751+0000 7fb4d3fff700 1 --2- 172.21.15.181:0/467939527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d4108410 0x7fb4d4104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:19.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.751+0000 7fb4d3fff700 1 -- 172.21.15.181:0/467939527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4d41023a0 msgr2=0x7fb4d4104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:19.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.751+0000 7fb4d3fff700 1 --2- 172.21.15.181:0/467939527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4d41023a0 0x7fb4d4104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:19.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.751+0000 7fb4d3fff700 1 -- 172.21.15.181:0/467939527 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d41a7ff0 con 0x7fb4d4102d60 2024-05-03T20:34:19.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.752+0000 7fb4d3fff700 1 --2- 172.21.15.181:0/467939527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4d4102d60 0x7fb4d41045c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000c9b0 tx=0x7fb4c000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:19.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.752+0000 7fb4d1ffb700 1 -- 172.21.15.181:0/467939527 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0007b00 con 0x7fb4d4102d60 2024-05-03T20:34:19.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.752+0000 7fb4daf1b700 1 -- 172.21.15.181:0/467939527 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d41a82e0 con 0x7fb4d4102d60 2024-05-03T20:34:19.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.752+0000 7fb4d1ffb700 1 -- 172.21.15.181:0/467939527 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4c0007ca0 con 0x7fb4d4102d60 2024-05-03T20:34:19.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.752+0000 7fb4d1ffb700 1 -- 172.21.15.181:0/467939527 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0018660 con 0x7fb4d4102d60 2024-05-03T20:34:19.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.752+0000 7fb4daf1b700 1 -- 172.21.15.181:0/467939527 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d41a8820 con 0x7fb4d4102d60 2024-05-03T20:34:19.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.753+0000 7fb4daf1b700 1 -- 172.21.15.181:0/467939527 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4d406a190 con 0x7fb4d4102d60 2024-05-03T20:34:19.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.754+0000 7fb4d1ffb700 1 -- 172.21.15.181:0/467939527 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fb4c000f450 con 0x7fb4d4102d60 2024-05-03T20:34:19.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.754+0000 7fb4d1ffb700 1 --2- 172.21.15.181:0/467939527 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb4c40b2910 0x7fb4c40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:19.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.755+0000 7fb4d8cb7700 1 --2- 172.21.15.181:0/467939527 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb4c40b2910 0x7fb4c40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:19.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.755+0000 7fb4d1ffb700 1 -- 172.21.15.181:0/467939527 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fb4c00e21d0 con 0x7fb4d4102d60 2024-05-03T20:34:19.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.755+0000 7fb4d8cb7700 1 --2- 172.21.15.181:0/467939527 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb4c40b2910 0x7fb4c40b4dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb4c80098e0 tx=0x7fb4c8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:19.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.759+0000 7fb4d1ffb700 1 -- 172.21.15.181:0/467939527 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb4c0014030 con 0x7fb4d4102d60 2024-05-03T20:34:19.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:19 smithi196 ceph-mon[32858]: pgmap v303: 97 pgs: 97 active+clean; 624 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-05-03T20:34:19.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:19 smithi204 ceph-mon[33595]: pgmap v303: 97 pgs: 97 active+clean; 624 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-05-03T20:34:19.961 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.962+0000 7fb4daf1b700 1 -- 172.21.15.181:0/467939527 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4d41a8ae0 con 0x7fb4d4102d60 2024-05-03T20:34:19.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:19 smithi181 ceph-mon[29801]: pgmap v303: 97 pgs: 97 active+clean; 624 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-05-03T20:34:19.963 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.965+0000 7fb4d1ffb700 1 -- 172.21.15.181:0/467939527 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22240 (secure 0 0 0) 0x7fb4c00a9f40 con 0x7fb4d4102d60 2024-05-03T20:34:19.965 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:34:19.965 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":81,"fsid":"1cd058ee-098b-11ef-bc97-c7b262605968","created":"2024-05-03T20:25:39.427815+0000","modified":"2024-05-03T20:33:42.488002+0000","last_up_change":"2024-05-03T20:32:38.854730+0000","last_in_change":"2024-05-03T20:32:18.787388+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T20:29:01.855187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T20:33:35.737273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T20:33:39.337556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5f594d97-10a4-4f77-aebc-c00dbe92a4af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6803","nonce":1683973278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6805","nonce":1683973278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6809","nonce":1683973278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6807","nonce":1683973278}]},"public_addr":"172.21.15.181:6803/1683973278","cluster_addr":"172.21.15.181:6805/1683973278","heartbeat_back_addr":"172.21.15.181:6809/1683973278","heartbeat_front_addr":"172.21.15.181:6807/1683973278","state":["exists","up"]},{"osd":1,"uuid":"9726d641-8930-4b45-9192-6ee2b6bfcdcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6801","nonce":2136873375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6803","nonce":2136873375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6807","nonce":2136873375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6805","nonce":2136873375}]},"public_addr":"172.21.15.196:6801/2136873375","cluster_addr":"172.21.15.196:6803/2136873375","heartbeat_back_addr":"172.21.15.196:6807/2136873375","heartbeat_front_addr":"172.21.15.196:6805/2136873375","state":["exists","up"]},{"osd":2,"uuid":"3d5573cc-37c1-4c3b-a61e-2363ef3ff964","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6801","nonce":3291673137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6803","nonce":3291673137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6807","nonce":3291673137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6805","nonce":3291673137}]},"public_addr":"172.21.15.204:6801/3291673137","cluster_addr":"172.21.15.204:6803/3291673137","heartbeat_back_addr":"172.21.15.204:6807/3291673137","heartbeat_front_addr":"172.21.15.204:6805/3291673137","state":["exists","up"]},{"osd":3,"uuid":"b9b205e3-a335-4771-9a66-55cfe6a8c31a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6811","nonce":29064971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6813","nonce":29064971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6817","nonce":29064971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6815","nonce":29064971}]},"public_addr":"172.21.15.181:6811/29064971","cluster_addr":"172.21.15.181:6813/29064971","heartbeat_back_addr":"172.21.15.181:6817/29064971","heartbeat_front_addr":"172.21.15.181:6815/29064971","state":["exists","up"]},{"osd":4,"uuid":"550f01ef-1947-4f9f-b759-34e3ea831839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6809","nonce":1145408995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6811","nonce":1145408995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6815","nonce":1145408995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6813","nonce":1145408995}]},"public_addr":"172.21.15.196:6809/1145408995","cluster_addr":"172.21.15.196:6811/1145408995","heartbeat_back_addr":"172.21.15.196:6815/1145408995","heartbeat_front_addr":"172.21.15.196:6813/1145408995","state":["exists","up"]},{"osd":5,"uuid":"6b74cb3a-f39a-4131-8793-1d5286856aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6809","nonce":855680559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6811","nonce":855680559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6815","nonce":855680559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6813","nonce":855680559}]},"public_addr":"172.21.15.204:6809/855680559","cluster_addr":"172.21.15.204:6811/855680559","heartbeat_back_addr":"172.21.15.204:6815/855680559","heartbeat_front_addr":"172.21.15.204:6813/855680559","state":["exists","up"]},{"osd":6,"uuid":"dbe19bdb-08eb-40aa-842b-0710963d4497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6819","nonce":1561969338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6821","nonce":1561969338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6825","nonce":1561969338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6823","nonce":1561969338}]},"public_addr":"172.21.15.181:6819/1561969338","cluster_addr":"172.21.15.181:6821/1561969338","heartbeat_back_addr":"172.21.15.181:6825/1561969338","heartbeat_front_addr":"172.21.15.181:6823/1561969338","state":["exists","up"]},{"osd":7,"uuid":"016c145c-ad67-42ac-9775-ca2840790abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6817","nonce":824890163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6819","nonce":824890163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6823","nonce":824890163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6821","nonce":824890163}]},"public_addr":"172.21.15.196:6817/824890163","cluster_addr":"172.21.15.196:6819/824890163","heartbeat_back_addr":"172.21.15.196:6823/824890163","heartbeat_front_addr":"172.21.15.196:6821/824890163","state":["exists","up"]},{"osd":8,"uuid":"10235638-c1f1-41d3-8e24-f8b2344bfeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6817","nonce":3566909552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6819","nonce":3566909552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6823","nonce":3566909552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6821","nonce":3566909552}]},"public_addr":"172.21.15.204:6817/3566909552","cluster_addr":"172.21.15.204:6819/3566909552","heartbeat_back_addr":"172.21.15.204:6823/3566909552","heartbeat_front_addr":"172.21.15.204:6821/3566909552","state":["exists","up"]},{"osd":9,"uuid":"96ce5aa6-e6a9-4d83-8562-589ac94c8cd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6827","nonce":241693472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6829","nonce":241693472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6833","nonce":241693472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6831","nonce":241693472}]},"public_addr":"172.21.15.181:6827/241693472","cluster_addr":"172.21.15.181:6829/241693472","heartbeat_back_addr":"172.21.15.181:6833/241693472","heartbeat_front_addr":"172.21.15.181:6831/241693472","state":["exists","up"]},{"osd":10,"uuid":"d6c9ffab-731c-46d5-88cd-5f21ba5acfa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6825","nonce":3980645995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6827","nonce":3980645995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6831","nonce":3980645995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6829","nonce":3980645995}]},"public_addr":"172.21.15.196:6825/3980645995","cluster_addr":"172.21.15.196:6827/3980645995","heartbeat_back_addr":"172.21.15.196:6831/3980645995","heartbeat_front_addr":"172.21.15.196:6829/3980645995","state":["exists","up"]},{"osd":11,"uuid":"3f58a2d6-bbbb-4e4d-bf83-2c886dd9484b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6825","nonce":488710574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6827","nonce":488710574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6831","nonce":488710574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6829","nonce":488710574}]},"public_addr":"172.21.15.204:6825/488710574","cluster_addr":"172.21.15.204:6827/488710574","heartbeat_back_addr":"172.21.15.204:6831/488710574","heartbeat_front_addr":"172.21.15.204:6829/488710574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:28:09.700091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:28:33.838568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:28:57.265536+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:29:21.720066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:29:45.480447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:30:09.558384+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:30:34.253331+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:30:57.986111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:31:22.470746+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:31:47.548762+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:32:11.285657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:32:36.888004+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6800/2389231557":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/3398276284":"2024-05-04T20:26:06.561713+0000","172.21.15.181:6801/2389231557":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/2327733609":"2024-05-04T20:26:06.561713+0000","172.21.15.181:0/628540759":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/3387163329":"2024-05-04T20:26:06.561713+0000","172.21.15.181:6800/1802984663":"2024-05-04T20:26:06.561713+0000","172.21.15.181:0/67924109":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/3693389657":"2024-05-04T20:26:33.766772+0000","172.21.15.181:6801/1802984663":"2024-05-04T20:26:06.561713+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T20:34:19.967 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.968+0000 7fb4daf1b700 1 -- 172.21.15.181:0/467939527 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb4c40b2910 msgr2=0x7fb4c40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:19.967 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.968+0000 7fb4daf1b700 1 --2- 172.21.15.181:0/467939527 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb4c40b2910 0x7fb4c40b4dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb4c80098e0 tx=0x7fb4c8005960 comp rx=0 tx=0).stop 2024-05-03T20:34:19.968 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.968+0000 7fb4daf1b700 1 -- 172.21.15.181:0/467939527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4d4102d60 msgr2=0x7fb4d41045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:19.968 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.968+0000 7fb4daf1b700 1 --2- 172.21.15.181:0/467939527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4d4102d60 0x7fb4d41045c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000c9b0 tx=0x7fb4c000ccc0 comp rx=0 tx=0).stop 2024-05-03T20:34:19.968 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.968+0000 7fb4daf1b700 1 -- 172.21.15.181:0/467939527 shutdown_connections 2024-05-03T20:34:19.968 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.968+0000 7fb4daf1b700 1 --2- 172.21.15.181:0/467939527 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb4c40b2910 0x7fb4c40b4dd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:19.968 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.968+0000 7fb4daf1b700 1 --2- 172.21.15.181:0/467939527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d4108410 0x7fb4d4104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:19.968 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.968+0000 7fb4daf1b700 1 --2- 172.21.15.181:0/467939527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4d41023a0 0x7fb4d4104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:19.968 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.968+0000 7fb4daf1b700 1 --2- 172.21.15.181:0/467939527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4d4102d60 0x7fb4d41045c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:19.969 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.968+0000 7fb4daf1b700 1 -- 172.21.15.181:0/467939527 >> 172.21.15.181:0/467939527 conn(0x7fb4d4079350 msgr2=0x7fb4d4101670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:19.969 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.968+0000 7fb4daf1b700 1 -- 172.21.15.181:0/467939527 shutdown_connections 2024-05-03T20:34:19.969 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:19.968+0000 7fb4daf1b700 1 -- 172.21.15.181:0/467939527 wait complete. 2024-05-03T20:34:20.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:20 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/467939527' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T20:34:20.790 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-03T20:34:20.791 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-03T20:34:20.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:20 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/467939527' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T20:34:20.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:20 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/467939527' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T20:34:21.030 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:34:21.826 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:21 smithi181 ceph-mon[29801]: pgmap v304: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-05-03T20:34:21.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:21 smithi196 ceph-mon[32858]: pgmap v304: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-05-03T20:34:21.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:21 smithi204 ceph-mon[33595]: pgmap v304: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-05-03T20:34:22.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.682+0000 7f567d350700 1 -- 172.21.15.181:0/4180269811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56781076e0 msgr2=0x7f567810f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:22.683 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.682+0000 7f567d350700 1 --2- 172.21.15.181:0/4180269811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56781076e0 0x7f567810f2b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f56700099c0 tx=0x7f5670009cd0 comp rx=0 tx=0).stop 2024-05-03T20:34:22.683 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.683+0000 7f567d350700 1 -- 172.21.15.181:0/4180269811 shutdown_connections 2024-05-03T20:34:22.683 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.683+0000 7f567d350700 1 --2- 172.21.15.181:0/4180269811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56781076e0 0x7f567810f2b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:22.683 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.683+0000 7f567d350700 1 --2- 172.21.15.181:0/4180269811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5678102690 0x7f5678102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:22.683 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.683+0000 7f567d350700 1 --2- 172.21.15.181:0/4180269811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5678103040 0x7f5678107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:22.683 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.683+0000 7f567d350700 1 -- 172.21.15.181:0/4180269811 >> 172.21.15.181:0/4180269811 conn(0x7f56780783f0 msgr2=0x7f5678078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:22.683 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.683+0000 7f567d350700 1 -- 172.21.15.181:0/4180269811 shutdown_connections 2024-05-03T20:34:22.683 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.683+0000 7f567d350700 1 -- 172.21.15.181:0/4180269811 wait complete. 2024-05-03T20:34:22.683 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.683+0000 7f567d350700 1 Processor -- start 2024-05-03T20:34:22.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.683+0000 7f567d350700 1 -- start start 2024-05-03T20:34:22.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.684+0000 7f567d350700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5678102690 0x7f567819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:22.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.684+0000 7f567d350700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5678103040 0x7f567819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:22.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.684+0000 7f567d350700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56781076e0 0x7f5678199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:22.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.684+0000 7f567d350700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5678112580 con 0x7f5678102690 2024-05-03T20:34:22.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.684+0000 7f567d350700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5678112400 con 0x7f5678103040 2024-05-03T20:34:22.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.684+0000 7f567d350700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5678112700 con 0x7f56781076e0 2024-05-03T20:34:22.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.684+0000 7f5676ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5678102690 0x7f567819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:22.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.684+0000 7f5676ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5678102690 0x7f567819f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48996/0 (socket says 172.21.15.181:48996) 2024-05-03T20:34:22.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.684+0000 7f5676ffd700 1 -- 172.21.15.181:0/2916246460 learned_addr learned my addr 172.21.15.181:0/2916246460 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:34:22.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.684+0000 7f56777fe700 1 --2- 172.21.15.181:0/2916246460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56781076e0 0x7f5678199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:22.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.684+0000 7f566e7fc700 1 --2- 172.21.15.181:0/2916246460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5678103040 0x7f567819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:22.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.684+0000 7f5676ffd700 1 -- 172.21.15.181:0/2916246460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56781076e0 msgr2=0x7f5678199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:22.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.684+0000 7f5676ffd700 1 --2- 172.21.15.181:0/2916246460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56781076e0 0x7f5678199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:22.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.684+0000 7f5676ffd700 1 -- 172.21.15.181:0/2916246460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5678103040 msgr2=0x7f567819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:22.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.684+0000 7f5676ffd700 1 --2- 172.21.15.181:0/2916246460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5678103040 0x7f567819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:22.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.684+0000 7f5676ffd700 1 -- 172.21.15.181:0/2916246460 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f567819a0c0 con 0x7f5678102690 2024-05-03T20:34:22.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.684+0000 7f56777fe700 1 --2- 172.21.15.181:0/2916246460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56781076e0 0x7f5678199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:34:22.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.684+0000 7f566e7fc700 1 --2- 172.21.15.181:0/2916246460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5678103040 0x7f567819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:34:22.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.684+0000 7f5676ffd700 1 --2- 172.21.15.181:0/2916246460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5678102690 0x7f567819f7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f56680060b0 tx=0x7f5668007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:22.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.684+0000 7f5674ff9700 1 -- 172.21.15.181:0/2916246460 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566801a070 con 0x7f5678102690 2024-05-03T20:34:22.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.684+0000 7f5674ff9700 1 -- 172.21.15.181:0/2916246460 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f566800de20 con 0x7f5678102690 2024-05-03T20:34:22.687 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.684+0000 7f567d350700 1 -- 172.21.15.181:0/2916246460 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f567819a3b0 con 0x7f5678102690 2024-05-03T20:34:22.687 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.684+0000 7f5674ff9700 1 -- 172.21.15.181:0/2916246460 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5668014720 con 0x7f5678102690 2024-05-03T20:34:22.687 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.684+0000 7f567d350700 1 -- 172.21.15.181:0/2916246460 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56781a6670 con 0x7f5678102690 2024-05-03T20:34:22.687 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.685+0000 7f567d350700 1 -- 172.21.15.181:0/2916246460 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f567806a190 con 0x7f5678102690 2024-05-03T20:34:22.688 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.687+0000 7f5674ff9700 1 -- 172.21.15.181:0/2916246460 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f566801d070 con 0x7f5678102690 2024-05-03T20:34:22.688 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.687+0000 7f5674ff9700 1 --2- 172.21.15.181:0/2916246460 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f56640b2910 0x7f56640b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:22.688 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.688+0000 7f566e7fc700 1 --2- 172.21.15.181:0/2916246460 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f56640b2910 0x7f56640b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:22.688 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.688+0000 7f5674ff9700 1 -- 172.21.15.181:0/2916246460 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f56680e20d0 con 0x7f5678102690 2024-05-03T20:34:22.688 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.688+0000 7f566e7fc700 1 --2- 172.21.15.181:0/2916246460 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f56640b2910 0x7f56640b4dd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5660005e90 tx=0x7f5660005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:22.690 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.691+0000 7f5674ff9700 1 -- 172.21.15.181:0/2916246460 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5668010070 con 0x7f5678102690 2024-05-03T20:34:22.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.894+0000 7f567d350700 1 -- 172.21.15.181:0/2916246460 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f56780638e0 con 0x7f56640b2910 2024-05-03T20:34:22.901 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.902+0000 7f5674ff9700 1 -- 172.21.15.181:0/2916246460 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258742 (secure 0 0 0) 0x7f56780638e0 con 0x7f56640b2910 2024-05-03T20:34:22.905 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:34:22.907 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.905+0000 7f567d350700 1 -- 172.21.15.181:0/2916246460 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f56640b2910 msgr2=0x7f56640b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:22.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.905+0000 7f567d350700 1 --2- 172.21.15.181:0/2916246460 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f56640b2910 0x7f56640b4dd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5660005e90 tx=0x7f5660005d20 comp rx=0 tx=0).stop 2024-05-03T20:34:22.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.905+0000 7f567d350700 1 -- 172.21.15.181:0/2916246460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5678102690 msgr2=0x7f567819f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:22.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.905+0000 7f567d350700 1 --2- 172.21.15.181:0/2916246460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5678102690 0x7f567819f7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f56680060b0 tx=0x7f5668007f90 comp rx=0 tx=0).stop 2024-05-03T20:34:22.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.905+0000 7f567d350700 1 -- 172.21.15.181:0/2916246460 shutdown_connections 2024-05-03T20:34:22.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.905+0000 7f567d350700 1 --2- 172.21.15.181:0/2916246460 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f56640b2910 0x7f56640b4dd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:22.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.905+0000 7f567d350700 1 --2- 172.21.15.181:0/2916246460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56781076e0 0x7f5678199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:22.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.905+0000 7f567d350700 1 --2- 172.21.15.181:0/2916246460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5678102690 0x7f567819f7a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:22.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.905+0000 7f567d350700 1 --2- 172.21.15.181:0/2916246460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5678103040 0x7f567819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:22.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.905+0000 7f567d350700 1 -- 172.21.15.181:0/2916246460 >> 172.21.15.181:0/2916246460 conn(0x7f56780783f0 msgr2=0x7f567806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:22.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.906+0000 7f567d350700 1 -- 172.21.15.181:0/2916246460 shutdown_connections 2024-05-03T20:34:22.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:22.906+0000 7f567d350700 1 -- 172.21.15.181:0/2916246460 wait complete. 2024-05-03T20:34:22.908 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2024-05-03T20:34:23.627 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-05-03T20:34:21.858947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647798,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":326,"num_read_kb":255,"num_write":363,"num_write_kb":2275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":299,"ondisk_log_size":299,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365720,"kb_used_data":12408,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576104,"statfs":{"total":1151940427776,"available":1151565930496,"internally_reserved":0,"allocated":12705792,"data_stored":6982398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27388,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":118,"num_read_kb":88,"num_write":21,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003007"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.699073+0000","last_change":"2024-05-03T20:33:40.713047+0000","last_active":"2024-05-03T20:33:42.699073+0000","last_peered":"2024-05-03T20:33:42.699073+0000","last_clean":"2024-05-03T20:33:42.699073+0000","last_became_active":"2024-05-03T20:33:40.712718+0000","last_became_peered":"2024-05-03T20:33:40.712718+0000","last_unstale":"2024-05-03T20:33:42.699073+0000","last_undegraded":"2024-05-03T20:33:42.699073+0000","last_fullsized":"2024-05-03T20:33:42.699073+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:55:24.816247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714353+0000","last_change":"2024-05-03T20:33:40.716139+0000","last_active":"2024-05-03T20:33:42.714353+0000","last_peered":"2024-05-03T20:33:42.714353+0000","last_clean":"2024-05-03T20:33:42.714353+0000","last_became_active":"2024-05-03T20:33:40.715808+0000","last_became_peered":"2024-05-03T20:33:40.715808+0000","last_unstale":"2024-05-03T20:33:42.714353+0000","last_undegraded":"2024-05-03T20:33:42.714353+0000","last_fullsized":"2024-05-03T20:33:42.714353+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:36:30.410799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714182+0000","last_change":"2024-05-03T20:33:40.712650+0000","last_active":"2024-05-03T20:33:42.714182+0000","last_peered":"2024-05-03T20:33:42.714182+0000","last_clean":"2024-05-03T20:33:42.714182+0000","last_became_active":"2024-05-03T20:33:40.712285+0000","last_became_peered":"2024-05-03T20:33:40.712285+0000","last_unstale":"2024-05-03T20:33:42.714182+0000","last_undegraded":"2024-05-03T20:33:42.714182+0000","last_fullsized":"2024-05-03T20:33:42.714182+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:10:25.162097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.716065+0000","last_change":"2024-05-03T20:33:40.716602+0000","last_active":"2024-05-03T20:33:42.716065+0000","last_peered":"2024-05-03T20:33:42.716065+0000","last_clean":"2024-05-03T20:33:42.716065+0000","last_became_active":"2024-05-03T20:33:40.716282+0000","last_became_peered":"2024-05-03T20:33:40.716282+0000","last_unstale":"2024-05-03T20:33:42.716065+0000","last_undegraded":"2024-05-03T20:33:42.716065+0000","last_fullsized":"2024-05-03T20:33:42.716065+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:15:27.687359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711476+0000","last_change":"2024-05-03T20:33:40.701780+0000","last_active":"2024-05-03T20:33:42.711476+0000","last_peered":"2024-05-03T20:33:42.711476+0000","last_clean":"2024-05-03T20:33:42.711476+0000","last_became_active":"2024-05-03T20:33:40.701447+0000","last_became_peered":"2024-05-03T20:33:40.701447+0000","last_unstale":"2024-05-03T20:33:42.711476+0000","last_undegraded":"2024-05-03T20:33:42.711476+0000","last_fullsized":"2024-05-03T20:33:42.711476+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:23:53.219489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.696819+0000","last_change":"2024-05-03T20:33:40.721249+0000","last_active":"2024-05-03T20:33:42.696819+0000","last_peered":"2024-05-03T20:33:42.696819+0000","last_clean":"2024-05-03T20:33:42.696819+0000","last_became_active":"2024-05-03T20:33:40.720838+0000","last_became_peered":"2024-05-03T20:33:40.720838+0000","last_unstale":"2024-05-03T20:33:42.696819+0000","last_undegraded":"2024-05-03T20:33:42.696819+0000","last_fullsized":"2024-05-03T20:33:42.696819+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:37:12.868435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714398+0000","last_change":"2024-05-03T20:33:40.708834+0000","last_active":"2024-05-03T20:33:42.714398+0000","last_peered":"2024-05-03T20:33:42.714398+0000","last_clean":"2024-05-03T20:33:42.714398+0000","last_became_active":"2024-05-03T20:33:40.708471+0000","last_became_peered":"2024-05-03T20:33:40.708471+0000","last_unstale":"2024-05-03T20:33:42.714398+0000","last_undegraded":"2024-05-03T20:33:42.714398+0000","last_fullsized":"2024-05-03T20:33:42.714398+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:54:21.297452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711471+0000","last_change":"2024-05-03T20:33:40.707078+0000","last_active":"2024-05-03T20:33:42.711471+0000","last_peered":"2024-05-03T20:33:42.711471+0000","last_clean":"2024-05-03T20:33:42.711471+0000","last_became_active":"2024-05-03T20:33:40.706724+0000","last_became_peered":"2024-05-03T20:33:40.706724+0000","last_unstale":"2024-05-03T20:33:42.711471+0000","last_undegraded":"2024-05-03T20:33:42.711471+0000","last_fullsized":"2024-05-03T20:33:42.711471+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:54:27.397305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.712540+0000","last_change":"2024-05-03T20:33:40.712580+0000","last_active":"2024-05-03T20:33:42.712540+0000","last_peered":"2024-05-03T20:33:42.712540+0000","last_clean":"2024-05-03T20:33:42.712540+0000","last_became_active":"2024-05-03T20:33:40.712252+0000","last_became_peered":"2024-05-03T20:33:40.712252+0000","last_unstale":"2024-05-03T20:33:42.712540+0000","last_undegraded":"2024-05-03T20:33:42.712540+0000","last_fullsized":"2024-05-03T20:33:42.712540+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:59:11.095956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.697280+0000","last_change":"2024-05-03T20:33:40.715924+0000","last_active":"2024-05-03T20:33:42.697280+0000","last_peered":"2024-05-03T20:33:42.697280+0000","last_clean":"2024-05-03T20:33:42.697280+0000","last_became_active":"2024-05-03T20:33:40.714562+0000","last_became_peered":"2024-05-03T20:33:40.714562+0000","last_unstale":"2024-05-03T20:33:42.697280+0000","last_undegraded":"2024-05-03T20:33:42.697280+0000","last_fullsized":"2024-05-03T20:33:42.697280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:52:42.146393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714835+0000","last_change":"2024-05-03T20:33:40.711383+0000","last_active":"2024-05-03T20:33:42.714835+0000","last_peered":"2024-05-03T20:33:42.714835+0000","last_clean":"2024-05-03T20:33:42.714835+0000","last_became_active":"2024-05-03T20:33:40.710870+0000","last_became_peered":"2024-05-03T20:33:40.710870+0000","last_unstale":"2024-05-03T20:33:42.714835+0000","last_undegraded":"2024-05-03T20:33:42.714835+0000","last_fullsized":"2024-05-03T20:33:42.714835+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:49:18.905954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.716023+0000","last_change":"2024-05-03T20:33:40.713749+0000","last_active":"2024-05-03T20:33:42.716023+0000","last_peered":"2024-05-03T20:33:42.716023+0000","last_clean":"2024-05-03T20:33:42.716023+0000","last_became_active":"2024-05-03T20:33:40.713359+0000","last_became_peered":"2024-05-03T20:33:40.713359+0000","last_unstale":"2024-05-03T20:33:42.716023+0000","last_undegraded":"2024-05-03T20:33:42.716023+0000","last_fullsized":"2024-05-03T20:33:42.716023+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:40:46.840267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.698641+0000","last_change":"2024-05-03T20:33:40.708434+0000","last_active":"2024-05-03T20:33:42.698641+0000","last_peered":"2024-05-03T20:33:42.698641+0000","last_clean":"2024-05-03T20:33:42.698641+0000","last_became_active":"2024-05-03T20:33:40.707825+0000","last_became_peered":"2024-05-03T20:33:40.707825+0000","last_unstale":"2024-05-03T20:33:42.698641+0000","last_undegraded":"2024-05-03T20:33:42.698641+0000","last_fullsized":"2024-05-03T20:33:42.698641+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:42:49.837828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714248+0000","last_change":"2024-05-03T20:33:40.714974+0000","last_active":"2024-05-03T20:33:42.714248+0000","last_peered":"2024-05-03T20:33:42.714248+0000","last_clean":"2024-05-03T20:33:42.714248+0000","last_became_active":"2024-05-03T20:33:40.714612+0000","last_became_peered":"2024-05-03T20:33:40.714612+0000","last_unstale":"2024-05-03T20:33:42.714248+0000","last_undegraded":"2024-05-03T20:33:42.714248+0000","last_fullsized":"2024-05-03T20:33:42.714248+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:34:40.766696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.713894+0000","last_change":"2024-05-03T20:33:40.713443+0000","last_active":"2024-05-03T20:33:42.713894+0000","last_peered":"2024-05-03T20:33:42.713894+0000","last_clean":"2024-05-03T20:33:42.713894+0000","last_became_active":"2024-05-03T20:33:40.712803+0000","last_became_peered":"2024-05-03T20:33:40.712803+0000","last_unstale":"2024-05-03T20:33:42.713894+0000","last_undegraded":"2024-05-03T20:33:42.713894+0000","last_fullsized":"2024-05-03T20:33:42.713894+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:26:46.942918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711859+0000","last_change":"2024-05-03T20:33:40.715667+0000","last_active":"2024-05-03T20:33:42.711859+0000","last_peered":"2024-05-03T20:33:42.711859+0000","last_clean":"2024-05-03T20:33:42.711859+0000","last_became_active":"2024-05-03T20:33:40.715291+0000","last_became_peered":"2024-05-03T20:33:40.715291+0000","last_unstale":"2024-05-03T20:33:42.711859+0000","last_undegraded":"2024-05-03T20:33:42.711859+0000","last_fullsized":"2024-05-03T20:33:42.711859+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:21:56.061022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.699187+0000","last_change":"2024-05-03T20:33:40.714495+0000","last_active":"2024-05-03T20:33:42.699187+0000","last_peered":"2024-05-03T20:33:42.699187+0000","last_clean":"2024-05-03T20:33:42.699187+0000","last_became_active":"2024-05-03T20:33:40.714176+0000","last_became_peered":"2024-05-03T20:33:40.714176+0000","last_unstale":"2024-05-03T20:33:42.699187+0000","last_undegraded":"2024-05-03T20:33:42.699187+0000","last_fullsized":"2024-05-03T20:33:42.699187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:26:34.299414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711138+0000","last_change":"2024-05-03T20:33:40.705512+0000","last_active":"2024-05-03T20:33:42.711138+0000","last_peered":"2024-05-03T20:33:42.711138+0000","last_clean":"2024-05-03T20:33:42.711138+0000","last_became_active":"2024-05-03T20:33:40.705187+0000","last_became_peered":"2024-05-03T20:33:40.705187+0000","last_unstale":"2024-05-03T20:33:42.711138+0000","last_undegraded":"2024-05-03T20:33:42.711138+0000","last_fullsized":"2024-05-03T20:33:42.711138+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:28:56.412004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.713998+0000","last_change":"2024-05-03T20:33:40.706780+0000","last_active":"2024-05-03T20:33:42.713998+0000","last_peered":"2024-05-03T20:33:42.713998+0000","last_clean":"2024-05-03T20:33:42.713998+0000","last_became_active":"2024-05-03T20:33:40.706439+0000","last_became_peered":"2024-05-03T20:33:40.706439+0000","last_unstale":"2024-05-03T20:33:42.713998+0000","last_undegraded":"2024-05-03T20:33:42.713998+0000","last_fullsized":"2024-05-03T20:33:42.713998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:43:03.182219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":59,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:34:19.522042+0000","last_change":"2024-05-03T20:33:37.680899+0000","last_active":"2024-05-03T20:34:19.522042+0000","last_peered":"2024-05-03T20:34:19.522042+0000","last_clean":"2024-05-03T20:34:19.522042+0000","last_became_active":"2024-05-03T20:33:37.680602+0000","last_became_peered":"2024-05-03T20:33:37.680602+0000","last_unstale":"2024-05-03T20:34:19.522042+0000","last_undegraded":"2024-05-03T20:34:19.522042+0000","last_fullsized":"2024-05-03T20:34:19.522042+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:39:50.865486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711881+0000","last_change":"2024-05-03T20:33:40.710869+0000","last_active":"2024-05-03T20:33:42.711881+0000","last_peered":"2024-05-03T20:33:42.711881+0000","last_clean":"2024-05-03T20:33:42.711881+0000","last_became_active":"2024-05-03T20:33:40.710552+0000","last_became_peered":"2024-05-03T20:33:40.710552+0000","last_unstale":"2024-05-03T20:33:42.711881+0000","last_undegraded":"2024-05-03T20:33:42.711881+0000","last_fullsized":"2024-05-03T20:33:42.711881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:18:59.947702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714427+0000","last_change":"2024-05-03T20:33:38.237387+0000","last_active":"2024-05-03T20:33:42.714427+0000","last_peered":"2024-05-03T20:33:42.714427+0000","last_clean":"2024-05-03T20:33:42.714427+0000","last_became_active":"2024-05-03T20:33:38.237029+0000","last_became_peered":"2024-05-03T20:33:38.237029+0000","last_unstale":"2024-05-03T20:33:42.714427+0000","last_undegraded":"2024-05-03T20:33:42.714427+0000","last_fullsized":"2024-05-03T20:33:42.714427+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:58:58.360973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714533+0000","last_change":"2024-05-03T20:33:40.712653+0000","last_active":"2024-05-03T20:33:42.714533+0000","last_peered":"2024-05-03T20:33:42.714533+0000","last_clean":"2024-05-03T20:33:42.714533+0000","last_became_active":"2024-05-03T20:33:40.712283+0000","last_became_peered":"2024-05-03T20:33:40.712283+0000","last_unstale":"2024-05-03T20:33:42.714533+0000","last_undegraded":"2024-05-03T20:33:42.714533+0000","last_fullsized":"2024-05-03T20:33:42.714533+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:28:56.388292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:58.493919+0000","last_change":"2024-05-03T20:33:37.683740+0000","last_active":"2024-05-03T20:33:58.493919+0000","last_peered":"2024-05-03T20:33:58.493919+0000","last_clean":"2024-05-03T20:33:58.493919+0000","last_became_active":"2024-05-03T20:33:37.683046+0000","last_became_peered":"2024-05-03T20:33:37.683046+0000","last_unstale":"2024-05-03T20:33:58.493919+0000","last_undegraded":"2024-05-03T20:33:58.493919+0000","last_fullsized":"2024-05-03T20:33:58.493919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:41:58.066741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.712015+0000","last_change":"2024-05-03T20:33:40.711274+0000","last_active":"2024-05-03T20:33:42.712015+0000","last_peered":"2024-05-03T20:33:42.712015+0000","last_clean":"2024-05-03T20:33:42.712015+0000","last_became_active":"2024-05-03T20:33:40.710917+0000","last_became_peered":"2024-05-03T20:33:40.710917+0000","last_unstale":"2024-05-03T20:33:42.712015+0000","last_undegraded":"2024-05-03T20:33:42.712015+0000","last_fullsized":"2024-05-03T20:33:42.712015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:22:55.969325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:57.485975+0000","last_change":"2024-05-03T20:33:37.681383+0000","last_active":"2024-05-03T20:33:57.485975+0000","last_peered":"2024-05-03T20:33:57.485975+0000","last_clean":"2024-05-03T20:33:57.485975+0000","last_became_active":"2024-05-03T20:33:37.680941+0000","last_became_peered":"2024-05-03T20:33:37.680941+0000","last_unstale":"2024-05-03T20:33:57.485975+0000","last_undegraded":"2024-05-03T20:33:57.485975+0000","last_fullsized":"2024-05-03T20:33:57.485975+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:03:26.083579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.712145+0000","last_change":"2024-05-03T20:33:40.711762+0000","last_active":"2024-05-03T20:33:42.712145+0000","last_peered":"2024-05-03T20:33:42.712145+0000","last_clean":"2024-05-03T20:33:42.712145+0000","last_became_active":"2024-05-03T20:33:40.710630+0000","last_became_peered":"2024-05-03T20:33:40.710630+0000","last_unstale":"2024-05-03T20:33:42.712145+0000","last_undegraded":"2024-05-03T20:33:42.712145+0000","last_fullsized":"2024-05-03T20:33:42.712145+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:42:23.728916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:34:15.518311+0000","last_change":"2024-05-03T20:33:37.685895+0000","last_active":"2024-05-03T20:34:15.518311+0000","last_peered":"2024-05-03T20:34:15.518311+0000","last_clean":"2024-05-03T20:34:15.518311+0000","last_became_active":"2024-05-03T20:33:37.685399+0000","last_became_peered":"2024-05-03T20:33:37.685399+0000","last_unstale":"2024-05-03T20:34:15.518311+0000","last_undegraded":"2024-05-03T20:34:15.518311+0000","last_fullsized":"2024-05-03T20:34:15.518311+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:09:29.932599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711262+0000","last_change":"2024-05-03T20:33:40.704967+0000","last_active":"2024-05-03T20:33:42.711262+0000","last_peered":"2024-05-03T20:33:42.711262+0000","last_clean":"2024-05-03T20:33:42.711262+0000","last_became_active":"2024-05-03T20:33:40.704583+0000","last_became_peered":"2024-05-03T20:33:40.704583+0000","last_unstale":"2024-05-03T20:33:42.711262+0000","last_undegraded":"2024-05-03T20:33:42.711262+0000","last_fullsized":"2024-05-03T20:33:42.711262+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:46:31.720865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:56.475217+0000","last_change":"2024-05-03T20:33:37.680798+0000","last_active":"2024-05-03T20:33:56.475217+0000","last_peered":"2024-05-03T20:33:56.475217+0000","last_clean":"2024-05-03T20:33:56.475217+0000","last_became_active":"2024-05-03T20:33:37.680344+0000","last_became_peered":"2024-05-03T20:33:37.680344+0000","last_unstale":"2024-05-03T20:33:56.475217+0000","last_undegraded":"2024-05-03T20:33:56.475217+0000","last_fullsized":"2024-05-03T20:33:56.475217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:26:32.037104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.712254+0000","last_change":"2024-05-03T20:33:40.710976+0000","last_active":"2024-05-03T20:33:42.712254+0000","last_peered":"2024-05-03T20:33:42.712254+0000","last_clean":"2024-05-03T20:33:42.712254+0000","last_became_active":"2024-05-03T20:33:40.710649+0000","last_became_peered":"2024-05-03T20:33:40.710649+0000","last_unstale":"2024-05-03T20:33:42.712254+0000","last_undegraded":"2024-05-03T20:33:42.712254+0000","last_fullsized":"2024-05-03T20:33:42.712254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:57:30.367412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:55.467898+0000","last_change":"2024-05-03T20:33:37.684230+0000","last_active":"2024-05-03T20:33:55.467898+0000","last_peered":"2024-05-03T20:33:55.467898+0000","last_clean":"2024-05-03T20:33:55.467898+0000","last_became_active":"2024-05-03T20:33:37.683844+0000","last_became_peered":"2024-05-03T20:33:37.683844+0000","last_unstale":"2024-05-03T20:33:55.467898+0000","last_undegraded":"2024-05-03T20:33:55.467898+0000","last_fullsized":"2024-05-03T20:33:55.467898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:46:23.417163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711997+0000","last_change":"2024-05-03T20:33:40.706554+0000","last_active":"2024-05-03T20:33:42.711997+0000","last_peered":"2024-05-03T20:33:42.711997+0000","last_clean":"2024-05-03T20:33:42.711997+0000","last_became_active":"2024-05-03T20:33:40.706142+0000","last_became_peered":"2024-05-03T20:33:40.706142+0000","last_unstale":"2024-05-03T20:33:42.711997+0000","last_undegraded":"2024-05-03T20:33:42.711997+0000","last_fullsized":"2024-05-03T20:33:42.711997+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:01:28.324024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:57.484669+0000","last_change":"2024-05-03T20:33:37.683497+0000","last_active":"2024-05-03T20:33:57.484669+0000","last_peered":"2024-05-03T20:33:57.484669+0000","last_clean":"2024-05-03T20:33:57.484669+0000","last_became_active":"2024-05-03T20:33:37.683178+0000","last_became_peered":"2024-05-03T20:33:37.683178+0000","last_unstale":"2024-05-03T20:33:57.484669+0000","last_undegraded":"2024-05-03T20:33:57.484669+0000","last_fullsized":"2024-05-03T20:33:57.484669+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:07:29.771786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711424+0000","last_change":"2024-05-03T20:33:40.705447+0000","last_active":"2024-05-03T20:33:42.711424+0000","last_peered":"2024-05-03T20:33:42.711424+0000","last_clean":"2024-05-03T20:33:42.711424+0000","last_became_active":"2024-05-03T20:33:40.705102+0000","last_became_peered":"2024-05-03T20:33:40.705102+0000","last_unstale":"2024-05-03T20:33:42.711424+0000","last_undegraded":"2024-05-03T20:33:42.711424+0000","last_fullsized":"2024-05-03T20:33:42.711424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:55:55.354498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":65,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:34:17.520062+0000","last_change":"2024-05-03T20:33:37.681449+0000","last_active":"2024-05-03T20:34:17.520062+0000","last_peered":"2024-05-03T20:34:17.520062+0000","last_clean":"2024-05-03T20:34:17.520062+0000","last_became_active":"2024-05-03T20:33:37.680798+0000","last_became_peered":"2024-05-03T20:33:37.680798+0000","last_unstale":"2024-05-03T20:34:17.520062+0000","last_undegraded":"2024-05-03T20:34:17.520062+0000","last_fullsized":"2024-05-03T20:34:17.520062+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:42:10.029261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.712325+0000","last_change":"2024-05-03T20:33:40.708872+0000","last_active":"2024-05-03T20:33:42.712325+0000","last_peered":"2024-05-03T20:33:42.712325+0000","last_clean":"2024-05-03T20:33:42.712325+0000","last_became_active":"2024-05-03T20:33:40.708535+0000","last_became_peered":"2024-05-03T20:33:40.708535+0000","last_unstale":"2024-05-03T20:33:42.712325+0000","last_undegraded":"2024-05-03T20:33:42.712325+0000","last_fullsized":"2024-05-03T20:33:42.712325+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:21:08.949065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:57.484290+0000","last_change":"2024-05-03T20:33:37.685425+0000","last_active":"2024-05-03T20:33:57.484290+0000","last_peered":"2024-05-03T20:33:57.484290+0000","last_clean":"2024-05-03T20:33:57.484290+0000","last_became_active":"2024-05-03T20:33:37.685032+0000","last_became_peered":"2024-05-03T20:33:37.685032+0000","last_unstale":"2024-05-03T20:33:57.484290+0000","last_undegraded":"2024-05-03T20:33:57.484290+0000","last_fullsized":"2024-05-03T20:33:57.484290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:14:35.380780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714435+0000","last_change":"2024-05-03T20:33:40.713991+0000","last_active":"2024-05-03T20:33:42.714435+0000","last_peered":"2024-05-03T20:33:42.714435+0000","last_clean":"2024-05-03T20:33:42.714435+0000","last_became_active":"2024-05-03T20:33:40.713661+0000","last_became_peered":"2024-05-03T20:33:40.713661+0000","last_unstale":"2024-05-03T20:33:42.714435+0000","last_undegraded":"2024-05-03T20:33:42.714435+0000","last_fullsized":"2024-05-03T20:33:42.714435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:08:20.818863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:56.474686+0000","last_change":"2024-05-03T20:33:38.237724+0000","last_active":"2024-05-03T20:33:56.474686+0000","last_peered":"2024-05-03T20:33:56.474686+0000","last_clean":"2024-05-03T20:33:56.474686+0000","last_became_active":"2024-05-03T20:33:38.237340+0000","last_became_peered":"2024-05-03T20:33:38.237340+0000","last_unstale":"2024-05-03T20:33:56.474686+0000","last_undegraded":"2024-05-03T20:33:56.474686+0000","last_fullsized":"2024-05-03T20:33:56.474686+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:13:35.567527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.712380+0000","last_change":"2024-05-03T20:33:40.707454+0000","last_active":"2024-05-03T20:33:42.712380+0000","last_peered":"2024-05-03T20:33:42.712380+0000","last_clean":"2024-05-03T20:33:42.712380+0000","last_became_active":"2024-05-03T20:33:40.707080+0000","last_became_peered":"2024-05-03T20:33:40.707080+0000","last_unstale":"2024-05-03T20:33:42.712380+0000","last_undegraded":"2024-05-03T20:33:42.712380+0000","last_fullsized":"2024-05-03T20:33:42.712380+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:12:18.432048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'27","reported_seq":95,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:34:17.522528+0000","last_change":"2024-05-03T20:33:37.681144+0000","last_active":"2024-05-03T20:34:17.522528+0000","last_peered":"2024-05-03T20:34:17.522528+0000","last_clean":"2024-05-03T20:34:17.522528+0000","last_became_active":"2024-05-03T20:33:37.680531+0000","last_became_peered":"2024-05-03T20:33:37.680531+0000","last_unstale":"2024-05-03T20:34:17.522528+0000","last_undegraded":"2024-05-03T20:34:17.522528+0000","last_fullsized":"2024-05-03T20:34:17.522528+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:45:53.536622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49601,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":47,"num_read_kb":71,"num_write":29,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711916+0000","last_change":"2024-05-03T20:33:40.711336+0000","last_active":"2024-05-03T20:33:42.711916+0000","last_peered":"2024-05-03T20:33:42.711916+0000","last_clean":"2024-05-03T20:33:42.711916+0000","last_became_active":"2024-05-03T20:33:40.710991+0000","last_became_peered":"2024-05-03T20:33:40.710991+0000","last_unstale":"2024-05-03T20:33:42.711916+0000","last_undegraded":"2024-05-03T20:33:42.711916+0000","last_fullsized":"2024-05-03T20:33:42.711916+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:59:44.729974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:58.493166+0000","last_change":"2024-05-03T20:33:37.682280+0000","last_active":"2024-05-03T20:33:58.493166+0000","last_peered":"2024-05-03T20:33:58.493166+0000","last_clean":"2024-05-03T20:33:58.493166+0000","last_became_active":"2024-05-03T20:33:37.681612+0000","last_became_peered":"2024-05-03T20:33:37.681612+0000","last_unstale":"2024-05-03T20:33:58.493166+0000","last_undegraded":"2024-05-03T20:33:58.493166+0000","last_fullsized":"2024-05-03T20:33:58.493166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:50:05.310560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.698280+0000","last_change":"2024-05-03T20:33:40.710309+0000","last_active":"2024-05-03T20:33:42.698280+0000","last_peered":"2024-05-03T20:33:42.698280+0000","last_clean":"2024-05-03T20:33:42.698280+0000","last_became_active":"2024-05-03T20:33:40.710015+0000","last_became_peered":"2024-05-03T20:33:40.710015+0000","last_unstale":"2024-05-03T20:33:42.698280+0000","last_undegraded":"2024-05-03T20:33:42.698280+0000","last_fullsized":"2024-05-03T20:33:42.698280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:05:04.042635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:57.484730+0000","last_change":"2024-05-03T20:33:37.679525+0000","last_active":"2024-05-03T20:33:57.484730+0000","last_peered":"2024-05-03T20:33:57.484730+0000","last_clean":"2024-05-03T20:33:57.484730+0000","last_became_active":"2024-05-03T20:33:37.678905+0000","last_became_peered":"2024-05-03T20:33:37.678905+0000","last_unstale":"2024-05-03T20:33:57.484730+0000","last_undegraded":"2024-05-03T20:33:57.484730+0000","last_fullsized":"2024-05-03T20:33:57.484730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:17:06.225255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714612+0000","last_change":"2024-05-03T20:33:40.717613+0000","last_active":"2024-05-03T20:33:42.714612+0000","last_peered":"2024-05-03T20:33:42.714612+0000","last_clean":"2024-05-03T20:33:42.714612+0000","last_became_active":"2024-05-03T20:33:40.717348+0000","last_became_peered":"2024-05-03T20:33:40.717348+0000","last_unstale":"2024-05-03T20:33:42.714612+0000","last_undegraded":"2024-05-03T20:33:42.714612+0000","last_fullsized":"2024-05-03T20:33:42.714612+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:20:31.493731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'6","reported_seq":50,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:34:15.517684+0000","last_change":"2024-05-03T20:33:37.683083+0000","last_active":"2024-05-03T20:34:15.517684+0000","last_peered":"2024-05-03T20:34:15.517684+0000","last_clean":"2024-05-03T20:34:15.517684+0000","last_became_active":"2024-05-03T20:33:37.682700+0000","last_became_peered":"2024-05-03T20:33:37.682700+0000","last_unstale":"2024-05-03T20:34:15.517684+0000","last_undegraded":"2024-05-03T20:34:15.517684+0000","last_fullsized":"2024-05-03T20:34:15.517684+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:47:18.294314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714220+0000","last_change":"2024-05-03T20:33:40.711711+0000","last_active":"2024-05-03T20:33:42.714220+0000","last_peered":"2024-05-03T20:33:42.714220+0000","last_clean":"2024-05-03T20:33:42.714220+0000","last_became_active":"2024-05-03T20:33:40.711116+0000","last_became_peered":"2024-05-03T20:33:40.711116+0000","last_unstale":"2024-05-03T20:33:42.714220+0000","last_undegraded":"2024-05-03T20:33:42.714220+0000","last_fullsized":"2024-05-03T20:33:42.714220+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:53:18.231575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:57.485691+0000","last_change":"2024-05-03T20:33:38.238005+0000","last_active":"2024-05-03T20:33:57.485691+0000","last_peered":"2024-05-03T20:33:57.485691+0000","last_clean":"2024-05-03T20:33:57.485691+0000","last_became_active":"2024-05-03T20:33:38.237636+0000","last_became_peered":"2024-05-03T20:33:38.237636+0000","last_unstale":"2024-05-03T20:33:57.485691+0000","last_undegraded":"2024-05-03T20:33:57.485691+0000","last_fullsized":"2024-05-03T20:33:57.485691+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:49:15.501715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711951+0000","last_change":"2024-05-03T20:33:40.716317+0000","last_active":"2024-05-03T20:33:42.711951+0000","last_peered":"2024-05-03T20:33:42.711951+0000","last_clean":"2024-05-03T20:33:42.711951+0000","last_became_active":"2024-05-03T20:33:40.715780+0000","last_became_peered":"2024-05-03T20:33:40.715780+0000","last_unstale":"2024-05-03T20:33:42.711951+0000","last_undegraded":"2024-05-03T20:33:42.711951+0000","last_fullsized":"2024-05-03T20:33:42.711951+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:21:46.186992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:56.462813+0000","last_change":"2024-05-03T20:33:37.679690+0000","last_active":"2024-05-03T20:33:56.462813+0000","last_peered":"2024-05-03T20:33:56.462813+0000","last_clean":"2024-05-03T20:33:56.462813+0000","last_became_active":"2024-05-03T20:33:37.679214+0000","last_became_peered":"2024-05-03T20:33:37.679214+0000","last_unstale":"2024-05-03T20:33:56.462813+0000","last_undegraded":"2024-05-03T20:33:56.462813+0000","last_fullsized":"2024-05-03T20:33:56.462813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:08:54.467850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.699036+0000","last_change":"2024-05-03T20:33:40.714800+0000","last_active":"2024-05-03T20:33:42.699036+0000","last_peered":"2024-05-03T20:33:42.699036+0000","last_clean":"2024-05-03T20:33:42.699036+0000","last_became_active":"2024-05-03T20:33:40.714417+0000","last_became_peered":"2024-05-03T20:33:40.714417+0000","last_unstale":"2024-05-03T20:33:42.699036+0000","last_undegraded":"2024-05-03T20:33:42.699036+0000","last_fullsized":"2024-05-03T20:33:42.699036+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:06:37.110378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.710999+0000","last_change":"2024-05-03T20:33:38.237270+0000","last_active":"2024-05-03T20:33:42.710999+0000","last_peered":"2024-05-03T20:33:42.710999+0000","last_clean":"2024-05-03T20:33:42.710999+0000","last_became_active":"2024-05-03T20:33:38.236917+0000","last_became_peered":"2024-05-03T20:33:38.236917+0000","last_unstale":"2024-05-03T20:33:42.710999+0000","last_undegraded":"2024-05-03T20:33:42.710999+0000","last_fullsized":"2024-05-03T20:33:42.710999+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:38:58.079905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:43.026647+0000","last_change":"2024-05-03T20:33:40.707368+0000","last_active":"2024-05-03T20:33:43.026647+0000","last_peered":"2024-05-03T20:33:43.026647+0000","last_clean":"2024-05-03T20:33:43.026647+0000","last_became_active":"2024-05-03T20:33:40.707049+0000","last_became_peered":"2024-05-03T20:33:40.707049+0000","last_unstale":"2024-05-03T20:33:43.026647+0000","last_undegraded":"2024-05-03T20:33:43.026647+0000","last_fullsized":"2024-05-03T20:33:43.026647+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:02:57.790142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'6","reported_seq":52,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:34:18.521304+0000","last_change":"2024-05-03T20:33:37.679007+0000","last_active":"2024-05-03T20:34:18.521304+0000","last_peered":"2024-05-03T20:34:18.521304+0000","last_clean":"2024-05-03T20:34:18.521304+0000","last_became_active":"2024-05-03T20:33:37.678609+0000","last_became_peered":"2024-05-03T20:33:37.678609+0000","last_unstale":"2024-05-03T20:34:18.521304+0000","last_undegraded":"2024-05-03T20:34:18.521304+0000","last_fullsized":"2024-05-03T20:34:18.521304+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:30:17.622905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.696804+0000","last_change":"2024-05-03T20:33:40.713820+0000","last_active":"2024-05-03T20:33:42.696804+0000","last_peered":"2024-05-03T20:33:42.696804+0000","last_clean":"2024-05-03T20:33:42.696804+0000","last_became_active":"2024-05-03T20:33:40.713487+0000","last_became_peered":"2024-05-03T20:33:40.713487+0000","last_unstale":"2024-05-03T20:33:42.696804+0000","last_undegraded":"2024-05-03T20:33:42.696804+0000","last_fullsized":"2024-05-03T20:33:42.696804+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:23:03.724180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714153+0000","last_change":"2024-05-03T20:33:40.716663+0000","last_active":"2024-05-03T20:33:42.714153+0000","last_peered":"2024-05-03T20:33:42.714153+0000","last_clean":"2024-05-03T20:33:42.714153+0000","last_became_active":"2024-05-03T20:33:40.716201+0000","last_became_peered":"2024-05-03T20:33:40.716201+0000","last_unstale":"2024-05-03T20:33:42.714153+0000","last_undegraded":"2024-05-03T20:33:42.714153+0000","last_fullsized":"2024-05-03T20:33:42.714153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:38:12.975351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:34:20.523430+0000","last_change":"2024-05-03T20:33:38.237285+0000","last_active":"2024-05-03T20:34:20.523430+0000","last_peered":"2024-05-03T20:34:20.523430+0000","last_clean":"2024-05-03T20:34:20.523430+0000","last_became_active":"2024-05-03T20:33:38.236825+0000","last_became_peered":"2024-05-03T20:33:38.236825+0000","last_unstale":"2024-05-03T20:34:20.523430+0000","last_undegraded":"2024-05-03T20:34:20.523430+0000","last_fullsized":"2024-05-03T20:34:20.523430+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:46:52.963953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.712414+0000","last_change":"2024-05-03T20:33:40.713271+0000","last_active":"2024-05-03T20:33:42.712414+0000","last_peered":"2024-05-03T20:33:42.712414+0000","last_clean":"2024-05-03T20:33:42.712414+0000","last_became_active":"2024-05-03T20:33:40.712972+0000","last_became_peered":"2024-05-03T20:33:40.712972+0000","last_unstale":"2024-05-03T20:33:42.712414+0000","last_undegraded":"2024-05-03T20:33:42.712414+0000","last_fullsized":"2024-05-03T20:33:42.712414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:39:46.762096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:58.493230+0000","last_change":"2024-05-03T20:33:37.682196+0000","last_active":"2024-05-03T20:33:58.493230+0000","last_peered":"2024-05-03T20:33:58.493230+0000","last_clean":"2024-05-03T20:33:58.493230+0000","last_became_active":"2024-05-03T20:33:37.681671+0000","last_became_peered":"2024-05-03T20:33:37.681671+0000","last_unstale":"2024-05-03T20:33:58.493230+0000","last_undegraded":"2024-05-03T20:33:58.493230+0000","last_fullsized":"2024-05-03T20:33:58.493230+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:53:26.866288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":97,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714588+0000","last_change":"2024-05-03T20:32:15.372604+0000","last_active":"2024-05-03T20:33:42.714588+0000","last_peered":"2024-05-03T20:33:42.714588+0000","last_clean":"2024-05-03T20:33:42.714588+0000","last_became_active":"2024-05-03T20:32:15.366768+0000","last_became_peered":"2024-05-03T20:32:15.366768+0000","last_unstale":"2024-05-03T20:33:42.714588+0000","last_undegraded":"2024-05-03T20:33:42.714588+0000","last_fullsized":"2024-05-03T20:33:42.714588+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:29:02.045902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:29:02.045902+0000","last_clean_scrub_stamp":"2024-05-03T20:29:02.045902+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:02:05.424270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.698260+0000","last_change":"2024-05-03T20:33:40.710085+0000","last_active":"2024-05-03T20:33:42.698260+0000","last_peered":"2024-05-03T20:33:42.698260+0000","last_clean":"2024-05-03T20:33:42.698260+0000","last_became_active":"2024-05-03T20:33:40.709765+0000","last_became_peered":"2024-05-03T20:33:40.709765+0000","last_unstale":"2024-05-03T20:33:42.698260+0000","last_undegraded":"2024-05-03T20:33:42.698260+0000","last_fullsized":"2024-05-03T20:33:42.698260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:52:02.339529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:55.467593+0000","last_change":"2024-05-03T20:33:37.679697+0000","last_active":"2024-05-03T20:33:55.467593+0000","last_peered":"2024-05-03T20:33:55.467593+0000","last_clean":"2024-05-03T20:33:55.467593+0000","last_became_active":"2024-05-03T20:33:37.679301+0000","last_became_peered":"2024-05-03T20:33:37.679301+0000","last_unstale":"2024-05-03T20:33:55.467593+0000","last_undegraded":"2024-05-03T20:33:55.467593+0000","last_fullsized":"2024-05-03T20:33:55.467593+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:17:11.201482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:57.486253+0000","last_change":"2024-05-03T20:33:38.239293+0000","last_active":"2024-05-03T20:33:57.486253+0000","last_peered":"2024-05-03T20:33:57.486253+0000","last_clean":"2024-05-03T20:33:57.486253+0000","last_became_active":"2024-05-03T20:33:38.238888+0000","last_became_peered":"2024-05-03T20:33:38.238888+0000","last_unstale":"2024-05-03T20:33:57.486253+0000","last_undegraded":"2024-05-03T20:33:57.486253+0000","last_fullsized":"2024-05-03T20:33:57.486253+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:41:06.397280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.698576+0000","last_change":"2024-05-03T20:33:40.710180+0000","last_active":"2024-05-03T20:33:42.698576+0000","last_peered":"2024-05-03T20:33:42.698576+0000","last_clean":"2024-05-03T20:33:42.698576+0000","last_became_active":"2024-05-03T20:33:40.709322+0000","last_became_peered":"2024-05-03T20:33:40.709322+0000","last_unstale":"2024-05-03T20:33:42.698576+0000","last_undegraded":"2024-05-03T20:33:42.698576+0000","last_fullsized":"2024-05-03T20:33:42.698576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:15:44.569110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'13","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:34:13.713611+0000","last_change":"2024-05-03T20:33:37.682422+0000","last_active":"2024-05-03T20:34:13.713611+0000","last_peered":"2024-05-03T20:34:13.713611+0000","last_clean":"2024-05-03T20:34:13.713611+0000","last_became_active":"2024-05-03T20:33:37.682079+0000","last_became_peered":"2024-05-03T20:33:37.682079+0000","last_unstale":"2024-05-03T20:34:13.713611+0000","last_undegraded":"2024-05-03T20:34:13.713611+0000","last_fullsized":"2024-05-03T20:34:13.713611+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:54:43.806768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711943+0000","last_change":"2024-05-03T20:33:40.715859+0000","last_active":"2024-05-03T20:33:42.711943+0000","last_peered":"2024-05-03T20:33:42.711943+0000","last_clean":"2024-05-03T20:33:42.711943+0000","last_became_active":"2024-05-03T20:33:40.715511+0000","last_became_peered":"2024-05-03T20:33:40.715511+0000","last_unstale":"2024-05-03T20:33:42.711943+0000","last_undegraded":"2024-05-03T20:33:42.711943+0000","last_fullsized":"2024-05-03T20:33:42.711943+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:27:59.317403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:57.485633+0000","last_change":"2024-05-03T20:33:37.679161+0000","last_active":"2024-05-03T20:33:57.485633+0000","last_peered":"2024-05-03T20:33:57.485633+0000","last_clean":"2024-05-03T20:33:57.485633+0000","last_became_active":"2024-05-03T20:33:37.678825+0000","last_became_peered":"2024-05-03T20:33:37.678825+0000","last_unstale":"2024-05-03T20:33:57.485633+0000","last_undegraded":"2024-05-03T20:33:57.485633+0000","last_fullsized":"2024-05-03T20:33:57.485633+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:05:01.203880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714331+0000","last_change":"2024-05-03T20:33:40.713287+0000","last_active":"2024-05-03T20:33:42.714331+0000","last_peered":"2024-05-03T20:33:42.714331+0000","last_clean":"2024-05-03T20:33:42.714331+0000","last_became_active":"2024-05-03T20:33:40.712962+0000","last_became_peered":"2024-05-03T20:33:40.712962+0000","last_unstale":"2024-05-03T20:33:42.714331+0000","last_undegraded":"2024-05-03T20:33:42.714331+0000","last_fullsized":"2024-05-03T20:33:42.714331+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:22:59.151645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:54.461104+0000","last_change":"2024-05-03T20:33:38.237996+0000","last_active":"2024-05-03T20:33:54.461104+0000","last_peered":"2024-05-03T20:33:54.461104+0000","last_clean":"2024-05-03T20:33:54.461104+0000","last_became_active":"2024-05-03T20:33:38.237552+0000","last_became_peered":"2024-05-03T20:33:38.237552+0000","last_unstale":"2024-05-03T20:33:54.461104+0000","last_undegraded":"2024-05-03T20:33:54.461104+0000","last_fullsized":"2024-05-03T20:33:54.461104+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:23:47.946259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711154+0000","last_change":"2024-05-03T20:33:40.709634+0000","last_active":"2024-05-03T20:33:42.711154+0000","last_peered":"2024-05-03T20:33:42.711154+0000","last_clean":"2024-05-03T20:33:42.711154+0000","last_became_active":"2024-05-03T20:33:40.709195+0000","last_became_peered":"2024-05-03T20:33:40.709195+0000","last_unstale":"2024-05-03T20:33:42.711154+0000","last_undegraded":"2024-05-03T20:33:42.711154+0000","last_fullsized":"2024-05-03T20:33:42.711154+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:24:30.796765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:57.484732+0000","last_change":"2024-05-03T20:33:37.677729+0000","last_active":"2024-05-03T20:33:57.484732+0000","last_peered":"2024-05-03T20:33:57.484732+0000","last_clean":"2024-05-03T20:33:57.484732+0000","last_became_active":"2024-05-03T20:33:37.677285+0000","last_became_peered":"2024-05-03T20:33:37.677285+0000","last_unstale":"2024-05-03T20:33:57.484732+0000","last_undegraded":"2024-05-03T20:33:57.484732+0000","last_fullsized":"2024-05-03T20:33:57.484732+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:01:46.289588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.698655+0000","last_change":"2024-05-03T20:33:40.708573+0000","last_active":"2024-05-03T20:33:42.698655+0000","last_peered":"2024-05-03T20:33:42.698655+0000","last_clean":"2024-05-03T20:33:42.698655+0000","last_became_active":"2024-05-03T20:33:40.708238+0000","last_became_peered":"2024-05-03T20:33:40.708238+0000","last_unstale":"2024-05-03T20:33:42.698655+0000","last_undegraded":"2024-05-03T20:33:42.698655+0000","last_fullsized":"2024-05-03T20:33:42.698655+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:21:06.975075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:56.475851+0000","last_change":"2024-05-03T20:33:37.683415+0000","last_active":"2024-05-03T20:33:56.475851+0000","last_peered":"2024-05-03T20:33:56.475851+0000","last_clean":"2024-05-03T20:33:56.475851+0000","last_became_active":"2024-05-03T20:33:37.682935+0000","last_became_peered":"2024-05-03T20:33:37.682935+0000","last_unstale":"2024-05-03T20:33:56.475851+0000","last_undegraded":"2024-05-03T20:33:56.475851+0000","last_fullsized":"2024-05-03T20:33:56.475851+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:26:17.977777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711994+0000","last_change":"2024-05-03T20:33:40.708116+0000","last_active":"2024-05-03T20:33:42.711994+0000","last_peered":"2024-05-03T20:33:42.711994+0000","last_clean":"2024-05-03T20:33:42.711994+0000","last_became_active":"2024-05-03T20:33:40.707625+0000","last_became_peered":"2024-05-03T20:33:40.707625+0000","last_unstale":"2024-05-03T20:33:42.711994+0000","last_undegraded":"2024-05-03T20:33:42.711994+0000","last_fullsized":"2024-05-03T20:33:42.711994+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:52:44.984252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:54.462116+0000","last_change":"2024-05-03T20:33:37.685132+0000","last_active":"2024-05-03T20:33:54.462116+0000","last_peered":"2024-05-03T20:33:54.462116+0000","last_clean":"2024-05-03T20:33:54.462116+0000","last_became_active":"2024-05-03T20:33:37.684169+0000","last_became_peered":"2024-05-03T20:33:37.684169+0000","last_unstale":"2024-05-03T20:33:54.462116+0000","last_undegraded":"2024-05-03T20:33:54.462116+0000","last_fullsized":"2024-05-03T20:33:54.462116+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:41:24.163383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.715212+0000","last_change":"2024-05-03T20:33:40.713622+0000","last_active":"2024-05-03T20:33:42.715212+0000","last_peered":"2024-05-03T20:33:42.715212+0000","last_clean":"2024-05-03T20:33:42.715212+0000","last_became_active":"2024-05-03T20:33:40.712467+0000","last_became_peered":"2024-05-03T20:33:40.712467+0000","last_unstale":"2024-05-03T20:33:42.715212+0000","last_undegraded":"2024-05-03T20:33:42.715212+0000","last_fullsized":"2024-05-03T20:33:42.715212+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:40:07.948287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'5","reported_seq":52,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:34:19.523277+0000","last_change":"2024-05-03T20:33:38.237790+0000","last_active":"2024-05-03T20:34:19.523277+0000","last_peered":"2024-05-03T20:34:19.523277+0000","last_clean":"2024-05-03T20:34:19.523277+0000","last_became_active":"2024-05-03T20:33:38.237327+0000","last_became_peered":"2024-05-03T20:33:38.237327+0000","last_unstale":"2024-05-03T20:34:19.523277+0000","last_undegraded":"2024-05-03T20:34:19.523277+0000","last_fullsized":"2024-05-03T20:34:19.523277+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:59:15.465519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":25,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.715061+0000","last_change":"2024-05-03T20:33:40.710127+0000","last_active":"2024-05-03T20:33:42.715061+0000","last_peered":"2024-05-03T20:33:42.715061+0000","last_clean":"2024-05-03T20:33:42.715061+0000","last_became_active":"2024-05-03T20:33:40.709743+0000","last_became_peered":"2024-05-03T20:33:40.709743+0000","last_unstale":"2024-05-03T20:33:42.715061+0000","last_undegraded":"2024-05-03T20:33:42.715061+0000","last_fullsized":"2024-05-03T20:33:42.715061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:01:48.638259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:57.484711+0000","last_change":"2024-05-03T20:33:37.680009+0000","last_active":"2024-05-03T20:33:57.484711+0000","last_peered":"2024-05-03T20:33:57.484711+0000","last_clean":"2024-05-03T20:33:57.484711+0000","last_became_active":"2024-05-03T20:33:37.679577+0000","last_became_peered":"2024-05-03T20:33:37.679577+0000","last_unstale":"2024-05-03T20:33:57.484711+0000","last_undegraded":"2024-05-03T20:33:57.484711+0000","last_fullsized":"2024-05-03T20:33:57.484711+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:13:01.801473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714845+0000","last_change":"2024-05-03T20:33:40.714713+0000","last_active":"2024-05-03T20:33:42.714845+0000","last_peered":"2024-05-03T20:33:42.714845+0000","last_clean":"2024-05-03T20:33:42.714845+0000","last_became_active":"2024-05-03T20:33:40.711743+0000","last_became_peered":"2024-05-03T20:33:40.711743+0000","last_unstale":"2024-05-03T20:33:42.714845+0000","last_undegraded":"2024-05-03T20:33:42.714845+0000","last_fullsized":"2024-05-03T20:33:42.714845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:57:42.748953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:57.484323+0000","last_change":"2024-05-03T20:33:37.682038+0000","last_active":"2024-05-03T20:33:57.484323+0000","last_peered":"2024-05-03T20:33:57.484323+0000","last_clean":"2024-05-03T20:33:57.484323+0000","last_became_active":"2024-05-03T20:33:37.681559+0000","last_became_peered":"2024-05-03T20:33:37.681559+0000","last_unstale":"2024-05-03T20:33:57.484323+0000","last_undegraded":"2024-05-03T20:33:57.484323+0000","last_fullsized":"2024-05-03T20:33:57.484323+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:12:49.532764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.715015+0000","last_change":"2024-05-03T20:33:40.708025+0000","last_active":"2024-05-03T20:33:42.715015+0000","last_peered":"2024-05-03T20:33:42.715015+0000","last_clean":"2024-05-03T20:33:42.715015+0000","last_became_active":"2024-05-03T20:33:40.707601+0000","last_became_peered":"2024-05-03T20:33:40.707601+0000","last_unstale":"2024-05-03T20:33:42.715015+0000","last_undegraded":"2024-05-03T20:33:42.715015+0000","last_fullsized":"2024-05-03T20:33:42.715015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:34:32.060247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.698962+0000","last_change":"2024-05-03T20:33:40.711728+0000","last_active":"2024-05-03T20:33:42.698962+0000","last_peered":"2024-05-03T20:33:42.698962+0000","last_clean":"2024-05-03T20:33:42.698962+0000","last_became_active":"2024-05-03T20:33:40.711367+0000","last_became_peered":"2024-05-03T20:33:40.711367+0000","last_unstale":"2024-05-03T20:33:42.698962+0000","last_undegraded":"2024-05-03T20:33:42.698962+0000","last_fullsized":"2024-05-03T20:33:42.698962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:07:17.956339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.713753+0000","last_change":"2024-05-03T20:33:40.716758+0000","last_active":"2024-05-03T20:33:42.713753+0000","last_peered":"2024-05-03T20:33:42.713753+0000","last_clean":"2024-05-03T20:33:42.713753+0000","last_became_active":"2024-05-03T20:33:40.716281+0000","last_became_peered":"2024-05-03T20:33:40.716281+0000","last_unstale":"2024-05-03T20:33:42.713753+0000","last_undegraded":"2024-05-03T20:33:42.713753+0000","last_fullsized":"2024-05-03T20:33:42.713753+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:59:40.795475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.712508+0000","last_change":"2024-05-03T20:33:40.705468+0000","last_active":"2024-05-03T20:33:42.712508+0000","last_peered":"2024-05-03T20:33:42.712508+0000","last_clean":"2024-05-03T20:33:42.712508+0000","last_became_active":"2024-05-03T20:33:40.705114+0000","last_became_peered":"2024-05-03T20:33:40.705114+0000","last_unstale":"2024-05-03T20:33:42.712508+0000","last_undegraded":"2024-05-03T20:33:42.712508+0000","last_fullsized":"2024-05-03T20:33:42.712508+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:20:05.549638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714387+0000","last_change":"2024-05-03T20:33:40.713282+0000","last_active":"2024-05-03T20:33:42.714387+0000","last_peered":"2024-05-03T20:33:42.714387+0000","last_clean":"2024-05-03T20:33:42.714387+0000","last_became_active":"2024-05-03T20:33:40.712674+0000","last_became_peered":"2024-05-03T20:33:40.712674+0000","last_unstale":"2024-05-03T20:33:42.714387+0000","last_undegraded":"2024-05-03T20:33:42.714387+0000","last_fullsized":"2024-05-03T20:33:42.714387+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:46:53.034439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.698677+0000","last_change":"2024-05-03T20:33:40.707870+0000","last_active":"2024-05-03T20:33:42.698677+0000","last_peered":"2024-05-03T20:33:42.698677+0000","last_clean":"2024-05-03T20:33:42.698677+0000","last_became_active":"2024-05-03T20:33:40.706849+0000","last_became_peered":"2024-05-03T20:33:40.706849+0000","last_unstale":"2024-05-03T20:33:42.698677+0000","last_undegraded":"2024-05-03T20:33:42.698677+0000","last_fullsized":"2024-05-03T20:33:42.698677+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:23:18.030508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.699007+0000","last_change":"2024-05-03T20:33:40.716365+0000","last_active":"2024-05-03T20:33:42.699007+0000","last_peered":"2024-05-03T20:33:42.699007+0000","last_clean":"2024-05-03T20:33:42.699007+0000","last_became_active":"2024-05-03T20:33:40.716010+0000","last_became_peered":"2024-05-03T20:33:40.716010+0000","last_unstale":"2024-05-03T20:33:42.699007+0000","last_undegraded":"2024-05-03T20:33:42.699007+0000","last_fullsized":"2024-05-03T20:33:42.699007+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:29:08.459822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.715739+0000","last_change":"2024-05-03T20:33:40.712105+0000","last_active":"2024-05-03T20:33:42.715739+0000","last_peered":"2024-05-03T20:33:42.715739+0000","last_clean":"2024-05-03T20:33:42.715739+0000","last_became_active":"2024-05-03T20:33:40.711753+0000","last_became_peered":"2024-05-03T20:33:40.711753+0000","last_unstale":"2024-05-03T20:33:42.715739+0000","last_undegraded":"2024-05-03T20:33:42.715739+0000","last_fullsized":"2024-05-03T20:33:42.715739+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:29:14.895475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711734+0000","last_change":"2024-05-03T20:33:40.713143+0000","last_active":"2024-05-03T20:33:42.711734+0000","last_peered":"2024-05-03T20:33:42.711734+0000","last_clean":"2024-05-03T20:33:42.711734+0000","last_became_active":"2024-05-03T20:33:40.712776+0000","last_became_peered":"2024-05-03T20:33:40.712776+0000","last_unstale":"2024-05-03T20:33:42.711734+0000","last_undegraded":"2024-05-03T20:33:42.711734+0000","last_fullsized":"2024-05-03T20:33:42.711734+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:43:23.345103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.713743+0000","last_change":"2024-05-03T20:33:40.706181+0000","last_active":"2024-05-03T20:33:42.713743+0000","last_peered":"2024-05-03T20:33:42.713743+0000","last_clean":"2024-05-03T20:33:42.713743+0000","last_became_active":"2024-05-03T20:33:40.705808+0000","last_became_peered":"2024-05-03T20:33:40.705808+0000","last_unstale":"2024-05-03T20:33:42.713743+0000","last_undegraded":"2024-05-03T20:33:42.713743+0000","last_fullsized":"2024-05-03T20:33:42.713743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:26:00.035916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711519+0000","last_change":"2024-05-03T20:33:40.708950+0000","last_active":"2024-05-03T20:33:42.711519+0000","last_peered":"2024-05-03T20:33:42.711519+0000","last_clean":"2024-05-03T20:33:42.711519+0000","last_became_active":"2024-05-03T20:33:40.708578+0000","last_became_peered":"2024-05-03T20:33:40.708578+0000","last_unstale":"2024-05-03T20:33:42.711519+0000","last_undegraded":"2024-05-03T20:33:42.711519+0000","last_fullsized":"2024-05-03T20:33:42.711519+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:35:12.767718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.712210+0000","last_change":"2024-05-03T20:33:40.715949+0000","last_active":"2024-05-03T20:33:42.712210+0000","last_peered":"2024-05-03T20:33:42.712210+0000","last_clean":"2024-05-03T20:33:42.712210+0000","last_became_active":"2024-05-03T20:33:40.715366+0000","last_became_peered":"2024-05-03T20:33:40.715366+0000","last_unstale":"2024-05-03T20:33:42.712210+0000","last_undegraded":"2024-05-03T20:33:42.712210+0000","last_fullsized":"2024-05-03T20:33:42.712210+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:32:54.485990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.698650+0000","last_change":"2024-05-03T20:33:40.710447+0000","last_active":"2024-05-03T20:33:42.698650+0000","last_peered":"2024-05-03T20:33:42.698650+0000","last_clean":"2024-05-03T20:33:42.698650+0000","last_became_active":"2024-05-03T20:33:40.710107+0000","last_became_peered":"2024-05-03T20:33:40.710107+0000","last_unstale":"2024-05-03T20:33:42.698650+0000","last_undegraded":"2024-05-03T20:33:42.698650+0000","last_fullsized":"2024-05-03T20:33:42.698650+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:47:43.678947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.699004+0000","last_change":"2024-05-03T20:33:40.714081+0000","last_active":"2024-05-03T20:33:42.699004+0000","last_peered":"2024-05-03T20:33:42.699004+0000","last_clean":"2024-05-03T20:33:42.699004+0000","last_became_active":"2024-05-03T20:33:40.713678+0000","last_became_peered":"2024-05-03T20:33:40.713678+0000","last_unstale":"2024-05-03T20:33:42.699004+0000","last_undegraded":"2024-05-03T20:33:42.699004+0000","last_fullsized":"2024-05-03T20:33:42.699004+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:24:30.128065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":57196,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":250,"num_read_kb":191,"num_write":238,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":171588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612630,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":876544,"data_stored":421109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808859,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1062130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972386,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":421039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168614,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":421417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397546,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28476,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716676,"statfs":{"total":95995035648,"available":95965876224,"internally_reserved":0,"allocated":1495040,"data_stored":1014717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561072,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":421147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":471661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":905216,"data_stored":422095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":954368,"data_stored":471881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117054,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":420581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920824,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":421008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757300,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1507328,"data_stored":1013613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":51744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":51964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":51728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-03T20:34:23.632 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-03T20:34:23.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:23 smithi196 ceph-mon[32858]: pgmap v305: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-05-03T20:34:23.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:23 smithi204 ceph-mon[33595]: pgmap v305: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-05-03T20:34:23.870 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:34:23.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:23 smithi181 ceph-mon[29801]: pgmap v305: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-05-03T20:34:24.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:24 smithi181 ceph-mon[29801]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T20:34:24.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:24 smithi196 ceph-mon[32858]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T20:34:24.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:24 smithi204 ceph-mon[33595]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T20:34:25.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.395+0000 7fa9bbd18700 1 -- 172.21.15.181:0/70425465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b4107760 msgr2=0x7fa9b410f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:25.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.395+0000 7fa9bbd18700 1 --2- 172.21.15.181:0/70425465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b4107760 0x7fa9b410f2e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00b0d0 tx=0x7fa9ac00b3e0 comp rx=0 tx=0).stop 2024-05-03T20:34:25.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.395+0000 7fa9bbd18700 1 -- 172.21.15.181:0/70425465 shutdown_connections 2024-05-03T20:34:25.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.395+0000 7fa9bbd18700 1 --2- 172.21.15.181:0/70425465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b4107760 0x7fa9b410f2e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:25.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.395+0000 7fa9bbd18700 1 --2- 172.21.15.181:0/70425465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9b4102e00 0x7fa9b41031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:25.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.395+0000 7fa9bbd18700 1 --2- 172.21.15.181:0/70425465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9b41037b0 0x7fa9b4107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:25.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.395+0000 7fa9bbd18700 1 -- 172.21.15.181:0/70425465 >> 172.21.15.181:0/70425465 conn(0x7fa9b4078a10 msgr2=0x7fa9b40ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:25.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.395+0000 7fa9bbd18700 1 -- 172.21.15.181:0/70425465 shutdown_connections 2024-05-03T20:34:25.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.395+0000 7fa9bbd18700 1 -- 172.21.15.181:0/70425465 wait complete. 2024-05-03T20:34:25.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.396+0000 7fa9bbd18700 1 Processor -- start 2024-05-03T20:34:25.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.396+0000 7fa9bbd18700 1 -- start start 2024-05-03T20:34:25.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.396+0000 7fa9bbd18700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9b4102e00 0x7fa9b40fde80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:25.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.396+0000 7fa9bbd18700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b41037b0 0x7fa9b40fe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:25.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.396+0000 7fa9bbd18700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9b4107760 0x7fa9b40fe900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:25.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.396+0000 7fa9bbd18700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b41125b0 con 0x7fa9b41037b0 2024-05-03T20:34:25.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.396+0000 7fa9bbd18700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b4112430 con 0x7fa9b4102e00 2024-05-03T20:34:25.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.396+0000 7fa9bbd18700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b4112730 con 0x7fa9b4107760 2024-05-03T20:34:25.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.396+0000 7fa9b92b3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b41037b0 0x7fa9b40fe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:25.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.396+0000 7fa9b92b3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b41037b0 0x7fa9b40fe3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49016/0 (socket says 172.21.15.181:49016) 2024-05-03T20:34:25.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.396+0000 7fa9b92b3700 1 -- 172.21.15.181:0/1274715222 learned_addr learned my addr 172.21.15.181:0/1274715222 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:34:25.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.396+0000 7fa9ba2b5700 1 --2- 172.21.15.181:0/1274715222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9b4107760 0x7fa9b40fe900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:25.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.396+0000 7fa9b9ab4700 1 --2- 172.21.15.181:0/1274715222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9b4102e00 0x7fa9b40fde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:25.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.396+0000 7fa9b92b3700 1 -- 172.21.15.181:0/1274715222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9b4107760 msgr2=0x7fa9b40fe900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:25.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.396+0000 7fa9b92b3700 1 --2- 172.21.15.181:0/1274715222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9b4107760 0x7fa9b40fe900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:25.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.396+0000 7fa9b92b3700 1 -- 172.21.15.181:0/1274715222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9b4102e00 msgr2=0x7fa9b40fde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:25.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.396+0000 7fa9b92b3700 1 --2- 172.21.15.181:0/1274715222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9b4102e00 0x7fa9b40fde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:25.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.396+0000 7fa9b92b3700 1 -- 172.21.15.181:0/1274715222 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b41a7fe0 con 0x7fa9b41037b0 2024-05-03T20:34:25.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.396+0000 7fa9ba2b5700 1 --2- 172.21.15.181:0/1274715222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9b4107760 0x7fa9b40fe900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:34:25.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.396+0000 7fa9b9ab4700 1 --2- 172.21.15.181:0/1274715222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9b4102e00 0x7fa9b40fde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:34:25.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.397+0000 7fa9b92b3700 1 --2- 172.21.15.181:0/1274715222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b41037b0 0x7fa9b40fe3c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa9b000c920 tx=0x7fa9b000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:25.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.397+0000 7fa9aaffd700 1 -- 172.21.15.181:0/1274715222 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0007b00 con 0x7fa9b41037b0 2024-05-03T20:34:25.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.397+0000 7fa9aaffd700 1 -- 172.21.15.181:0/1274715222 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b0004ce0 con 0x7fa9b41037b0 2024-05-03T20:34:25.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.397+0000 7fa9bbd18700 1 -- 172.21.15.181:0/1274715222 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b41a82d0 con 0x7fa9b41037b0 2024-05-03T20:34:25.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.397+0000 7fa9aaffd700 1 -- 172.21.15.181:0/1274715222 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0005650 con 0x7fa9b41037b0 2024-05-03T20:34:25.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.397+0000 7fa9bbd18700 1 -- 172.21.15.181:0/1274715222 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b41a8810 con 0x7fa9b41037b0 2024-05-03T20:34:25.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.398+0000 7fa9bbd18700 1 -- 172.21.15.181:0/1274715222 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9b406a190 con 0x7fa9b41037b0 2024-05-03T20:34:25.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.399+0000 7fa9aaffd700 1 -- 172.21.15.181:0/1274715222 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fa9b0004750 con 0x7fa9b41037b0 2024-05-03T20:34:25.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.400+0000 7fa9aaffd700 1 --2- 172.21.15.181:0/1274715222 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fa9a00b2910 0x7fa9a00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:25.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.400+0000 7fa9b9ab4700 1 --2- 172.21.15.181:0/1274715222 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fa9a00b2910 0x7fa9a00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:25.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.400+0000 7fa9aaffd700 1 -- 172.21.15.181:0/1274715222 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fa9b00e2240 con 0x7fa9b41037b0 2024-05-03T20:34:25.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.400+0000 7fa9b9ab4700 1 --2- 172.21.15.181:0/1274715222 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fa9a00b2910 0x7fa9a00b4dd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4007900 tx=0x7fa9a4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:25.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.403+0000 7fa9aaffd700 1 -- 172.21.15.181:0/1274715222 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fa9b0014030 con 0x7fa9b41037b0 2024-05-03T20:34:25.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.607+0000 7fa9bbd18700 1 -- 172.21.15.181:0/1274715222 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa9b40638e0 con 0x7fa9a00b2910 2024-05-03T20:34:25.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.614+0000 7fa9aaffd700 1 -- 172.21.15.181:0/1274715222 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258743 (secure 0 0 0) 0x7fa9b40638e0 con 0x7fa9a00b2910 2024-05-03T20:34:25.617 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:34:25.619 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.617+0000 7fa9bbd18700 1 -- 172.21.15.181:0/1274715222 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fa9a00b2910 msgr2=0x7fa9a00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:25.619 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.617+0000 7fa9bbd18700 1 --2- 172.21.15.181:0/1274715222 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fa9a00b2910 0x7fa9a00b4dd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4007900 tx=0x7fa9a4008040 comp rx=0 tx=0).stop 2024-05-03T20:34:25.619 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.617+0000 7fa9bbd18700 1 -- 172.21.15.181:0/1274715222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b41037b0 msgr2=0x7fa9b40fe3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:25.619 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.617+0000 7fa9bbd18700 1 --2- 172.21.15.181:0/1274715222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b41037b0 0x7fa9b40fe3c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa9b000c920 tx=0x7fa9b000cce0 comp rx=0 tx=0).stop 2024-05-03T20:34:25.619 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.617+0000 7fa9bbd18700 1 -- 172.21.15.181:0/1274715222 shutdown_connections 2024-05-03T20:34:25.619 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.617+0000 7fa9bbd18700 1 --2- 172.21.15.181:0/1274715222 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fa9a00b2910 0x7fa9a00b4dd0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:25.619 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.617+0000 7fa9bbd18700 1 --2- 172.21.15.181:0/1274715222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9b4107760 0x7fa9b40fe900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:25.619 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.617+0000 7fa9bbd18700 1 --2- 172.21.15.181:0/1274715222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9b4102e00 0x7fa9b40fde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:25.619 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.617+0000 7fa9bbd18700 1 --2- 172.21.15.181:0/1274715222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b41037b0 0x7fa9b40fe3c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:25.620 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.617+0000 7fa9bbd18700 1 -- 172.21.15.181:0/1274715222 >> 172.21.15.181:0/1274715222 conn(0x7fa9b4078a10 msgr2=0x7fa9b4101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:25.620 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.617+0000 7fa9bbd18700 1 -- 172.21.15.181:0/1274715222 shutdown_connections 2024-05-03T20:34:25.620 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:25.617+0000 7fa9bbd18700 1 -- 172.21.15.181:0/1274715222 wait complete. 2024-05-03T20:34:25.620 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2024-05-03T20:34:25.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:25 smithi196 ceph-mon[32858]: pgmap v306: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-05-03T20:34:25.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:25 smithi204 ceph-mon[33595]: pgmap v306: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-05-03T20:34:25.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:25 smithi181 ceph-mon[29801]: pgmap v306: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-05-03T20:34:26.277 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-05-03T20:34:23.859472+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647798,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":356,"num_read_kb":270,"num_write":363,"num_write_kb":2275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":299,"ondisk_log_size":299,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365720,"kb_used_data":12408,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576104,"statfs":{"total":1151940427776,"available":1151565930496,"internally_reserved":0,"allocated":12705792,"data_stored":6982398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27388,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":133,"num_read_kb":93,"num_write":17,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003069"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.699073+0000","last_change":"2024-05-03T20:33:40.713047+0000","last_active":"2024-05-03T20:33:42.699073+0000","last_peered":"2024-05-03T20:33:42.699073+0000","last_clean":"2024-05-03T20:33:42.699073+0000","last_became_active":"2024-05-03T20:33:40.712718+0000","last_became_peered":"2024-05-03T20:33:40.712718+0000","last_unstale":"2024-05-03T20:33:42.699073+0000","last_undegraded":"2024-05-03T20:33:42.699073+0000","last_fullsized":"2024-05-03T20:33:42.699073+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:55:24.816247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714353+0000","last_change":"2024-05-03T20:33:40.716139+0000","last_active":"2024-05-03T20:33:42.714353+0000","last_peered":"2024-05-03T20:33:42.714353+0000","last_clean":"2024-05-03T20:33:42.714353+0000","last_became_active":"2024-05-03T20:33:40.715808+0000","last_became_peered":"2024-05-03T20:33:40.715808+0000","last_unstale":"2024-05-03T20:33:42.714353+0000","last_undegraded":"2024-05-03T20:33:42.714353+0000","last_fullsized":"2024-05-03T20:33:42.714353+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:36:30.410799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714182+0000","last_change":"2024-05-03T20:33:40.712650+0000","last_active":"2024-05-03T20:33:42.714182+0000","last_peered":"2024-05-03T20:33:42.714182+0000","last_clean":"2024-05-03T20:33:42.714182+0000","last_became_active":"2024-05-03T20:33:40.712285+0000","last_became_peered":"2024-05-03T20:33:40.712285+0000","last_unstale":"2024-05-03T20:33:42.714182+0000","last_undegraded":"2024-05-03T20:33:42.714182+0000","last_fullsized":"2024-05-03T20:33:42.714182+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:10:25.162097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.716065+0000","last_change":"2024-05-03T20:33:40.716602+0000","last_active":"2024-05-03T20:33:42.716065+0000","last_peered":"2024-05-03T20:33:42.716065+0000","last_clean":"2024-05-03T20:33:42.716065+0000","last_became_active":"2024-05-03T20:33:40.716282+0000","last_became_peered":"2024-05-03T20:33:40.716282+0000","last_unstale":"2024-05-03T20:33:42.716065+0000","last_undegraded":"2024-05-03T20:33:42.716065+0000","last_fullsized":"2024-05-03T20:33:42.716065+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:15:27.687359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711476+0000","last_change":"2024-05-03T20:33:40.701780+0000","last_active":"2024-05-03T20:33:42.711476+0000","last_peered":"2024-05-03T20:33:42.711476+0000","last_clean":"2024-05-03T20:33:42.711476+0000","last_became_active":"2024-05-03T20:33:40.701447+0000","last_became_peered":"2024-05-03T20:33:40.701447+0000","last_unstale":"2024-05-03T20:33:42.711476+0000","last_undegraded":"2024-05-03T20:33:42.711476+0000","last_fullsized":"2024-05-03T20:33:42.711476+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:23:53.219489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.696819+0000","last_change":"2024-05-03T20:33:40.721249+0000","last_active":"2024-05-03T20:33:42.696819+0000","last_peered":"2024-05-03T20:33:42.696819+0000","last_clean":"2024-05-03T20:33:42.696819+0000","last_became_active":"2024-05-03T20:33:40.720838+0000","last_became_peered":"2024-05-03T20:33:40.720838+0000","last_unstale":"2024-05-03T20:33:42.696819+0000","last_undegraded":"2024-05-03T20:33:42.696819+0000","last_fullsized":"2024-05-03T20:33:42.696819+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:37:12.868435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714398+0000","last_change":"2024-05-03T20:33:40.708834+0000","last_active":"2024-05-03T20:33:42.714398+0000","last_peered":"2024-05-03T20:33:42.714398+0000","last_clean":"2024-05-03T20:33:42.714398+0000","last_became_active":"2024-05-03T20:33:40.708471+0000","last_became_peered":"2024-05-03T20:33:40.708471+0000","last_unstale":"2024-05-03T20:33:42.714398+0000","last_undegraded":"2024-05-03T20:33:42.714398+0000","last_fullsized":"2024-05-03T20:33:42.714398+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:54:21.297452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711471+0000","last_change":"2024-05-03T20:33:40.707078+0000","last_active":"2024-05-03T20:33:42.711471+0000","last_peered":"2024-05-03T20:33:42.711471+0000","last_clean":"2024-05-03T20:33:42.711471+0000","last_became_active":"2024-05-03T20:33:40.706724+0000","last_became_peered":"2024-05-03T20:33:40.706724+0000","last_unstale":"2024-05-03T20:33:42.711471+0000","last_undegraded":"2024-05-03T20:33:42.711471+0000","last_fullsized":"2024-05-03T20:33:42.711471+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:54:27.397305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.712540+0000","last_change":"2024-05-03T20:33:40.712580+0000","last_active":"2024-05-03T20:33:42.712540+0000","last_peered":"2024-05-03T20:33:42.712540+0000","last_clean":"2024-05-03T20:33:42.712540+0000","last_became_active":"2024-05-03T20:33:40.712252+0000","last_became_peered":"2024-05-03T20:33:40.712252+0000","last_unstale":"2024-05-03T20:33:42.712540+0000","last_undegraded":"2024-05-03T20:33:42.712540+0000","last_fullsized":"2024-05-03T20:33:42.712540+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:59:11.095956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.697280+0000","last_change":"2024-05-03T20:33:40.715924+0000","last_active":"2024-05-03T20:33:42.697280+0000","last_peered":"2024-05-03T20:33:42.697280+0000","last_clean":"2024-05-03T20:33:42.697280+0000","last_became_active":"2024-05-03T20:33:40.714562+0000","last_became_peered":"2024-05-03T20:33:40.714562+0000","last_unstale":"2024-05-03T20:33:42.697280+0000","last_undegraded":"2024-05-03T20:33:42.697280+0000","last_fullsized":"2024-05-03T20:33:42.697280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:52:42.146393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714835+0000","last_change":"2024-05-03T20:33:40.711383+0000","last_active":"2024-05-03T20:33:42.714835+0000","last_peered":"2024-05-03T20:33:42.714835+0000","last_clean":"2024-05-03T20:33:42.714835+0000","last_became_active":"2024-05-03T20:33:40.710870+0000","last_became_peered":"2024-05-03T20:33:40.710870+0000","last_unstale":"2024-05-03T20:33:42.714835+0000","last_undegraded":"2024-05-03T20:33:42.714835+0000","last_fullsized":"2024-05-03T20:33:42.714835+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:49:18.905954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.716023+0000","last_change":"2024-05-03T20:33:40.713749+0000","last_active":"2024-05-03T20:33:42.716023+0000","last_peered":"2024-05-03T20:33:42.716023+0000","last_clean":"2024-05-03T20:33:42.716023+0000","last_became_active":"2024-05-03T20:33:40.713359+0000","last_became_peered":"2024-05-03T20:33:40.713359+0000","last_unstale":"2024-05-03T20:33:42.716023+0000","last_undegraded":"2024-05-03T20:33:42.716023+0000","last_fullsized":"2024-05-03T20:33:42.716023+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:40:46.840267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.698641+0000","last_change":"2024-05-03T20:33:40.708434+0000","last_active":"2024-05-03T20:33:42.698641+0000","last_peered":"2024-05-03T20:33:42.698641+0000","last_clean":"2024-05-03T20:33:42.698641+0000","last_became_active":"2024-05-03T20:33:40.707825+0000","last_became_peered":"2024-05-03T20:33:40.707825+0000","last_unstale":"2024-05-03T20:33:42.698641+0000","last_undegraded":"2024-05-03T20:33:42.698641+0000","last_fullsized":"2024-05-03T20:33:42.698641+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:42:49.837828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714248+0000","last_change":"2024-05-03T20:33:40.714974+0000","last_active":"2024-05-03T20:33:42.714248+0000","last_peered":"2024-05-03T20:33:42.714248+0000","last_clean":"2024-05-03T20:33:42.714248+0000","last_became_active":"2024-05-03T20:33:40.714612+0000","last_became_peered":"2024-05-03T20:33:40.714612+0000","last_unstale":"2024-05-03T20:33:42.714248+0000","last_undegraded":"2024-05-03T20:33:42.714248+0000","last_fullsized":"2024-05-03T20:33:42.714248+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:34:40.766696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.713894+0000","last_change":"2024-05-03T20:33:40.713443+0000","last_active":"2024-05-03T20:33:42.713894+0000","last_peered":"2024-05-03T20:33:42.713894+0000","last_clean":"2024-05-03T20:33:42.713894+0000","last_became_active":"2024-05-03T20:33:40.712803+0000","last_became_peered":"2024-05-03T20:33:40.712803+0000","last_unstale":"2024-05-03T20:33:42.713894+0000","last_undegraded":"2024-05-03T20:33:42.713894+0000","last_fullsized":"2024-05-03T20:33:42.713894+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:26:46.942918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711859+0000","last_change":"2024-05-03T20:33:40.715667+0000","last_active":"2024-05-03T20:33:42.711859+0000","last_peered":"2024-05-03T20:33:42.711859+0000","last_clean":"2024-05-03T20:33:42.711859+0000","last_became_active":"2024-05-03T20:33:40.715291+0000","last_became_peered":"2024-05-03T20:33:40.715291+0000","last_unstale":"2024-05-03T20:33:42.711859+0000","last_undegraded":"2024-05-03T20:33:42.711859+0000","last_fullsized":"2024-05-03T20:33:42.711859+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:21:56.061022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.699187+0000","last_change":"2024-05-03T20:33:40.714495+0000","last_active":"2024-05-03T20:33:42.699187+0000","last_peered":"2024-05-03T20:33:42.699187+0000","last_clean":"2024-05-03T20:33:42.699187+0000","last_became_active":"2024-05-03T20:33:40.714176+0000","last_became_peered":"2024-05-03T20:33:40.714176+0000","last_unstale":"2024-05-03T20:33:42.699187+0000","last_undegraded":"2024-05-03T20:33:42.699187+0000","last_fullsized":"2024-05-03T20:33:42.699187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:26:34.299414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711138+0000","last_change":"2024-05-03T20:33:40.705512+0000","last_active":"2024-05-03T20:33:42.711138+0000","last_peered":"2024-05-03T20:33:42.711138+0000","last_clean":"2024-05-03T20:33:42.711138+0000","last_became_active":"2024-05-03T20:33:40.705187+0000","last_became_peered":"2024-05-03T20:33:40.705187+0000","last_unstale":"2024-05-03T20:33:42.711138+0000","last_undegraded":"2024-05-03T20:33:42.711138+0000","last_fullsized":"2024-05-03T20:33:42.711138+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:28:56.412004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.713998+0000","last_change":"2024-05-03T20:33:40.706780+0000","last_active":"2024-05-03T20:33:42.713998+0000","last_peered":"2024-05-03T20:33:42.713998+0000","last_clean":"2024-05-03T20:33:42.713998+0000","last_became_active":"2024-05-03T20:33:40.706439+0000","last_became_peered":"2024-05-03T20:33:40.706439+0000","last_unstale":"2024-05-03T20:33:42.713998+0000","last_undegraded":"2024-05-03T20:33:42.713998+0000","last_fullsized":"2024-05-03T20:33:42.713998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:43:03.182219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":59,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:34:19.522042+0000","last_change":"2024-05-03T20:33:37.680899+0000","last_active":"2024-05-03T20:34:19.522042+0000","last_peered":"2024-05-03T20:34:19.522042+0000","last_clean":"2024-05-03T20:34:19.522042+0000","last_became_active":"2024-05-03T20:33:37.680602+0000","last_became_peered":"2024-05-03T20:33:37.680602+0000","last_unstale":"2024-05-03T20:34:19.522042+0000","last_undegraded":"2024-05-03T20:34:19.522042+0000","last_fullsized":"2024-05-03T20:34:19.522042+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:39:50.865486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711881+0000","last_change":"2024-05-03T20:33:40.710869+0000","last_active":"2024-05-03T20:33:42.711881+0000","last_peered":"2024-05-03T20:33:42.711881+0000","last_clean":"2024-05-03T20:33:42.711881+0000","last_became_active":"2024-05-03T20:33:40.710552+0000","last_became_peered":"2024-05-03T20:33:40.710552+0000","last_unstale":"2024-05-03T20:33:42.711881+0000","last_undegraded":"2024-05-03T20:33:42.711881+0000","last_fullsized":"2024-05-03T20:33:42.711881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:18:59.947702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714427+0000","last_change":"2024-05-03T20:33:38.237387+0000","last_active":"2024-05-03T20:33:42.714427+0000","last_peered":"2024-05-03T20:33:42.714427+0000","last_clean":"2024-05-03T20:33:42.714427+0000","last_became_active":"2024-05-03T20:33:38.237029+0000","last_became_peered":"2024-05-03T20:33:38.237029+0000","last_unstale":"2024-05-03T20:33:42.714427+0000","last_undegraded":"2024-05-03T20:33:42.714427+0000","last_fullsized":"2024-05-03T20:33:42.714427+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:58:58.360973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714533+0000","last_change":"2024-05-03T20:33:40.712653+0000","last_active":"2024-05-03T20:33:42.714533+0000","last_peered":"2024-05-03T20:33:42.714533+0000","last_clean":"2024-05-03T20:33:42.714533+0000","last_became_active":"2024-05-03T20:33:40.712283+0000","last_became_peered":"2024-05-03T20:33:40.712283+0000","last_unstale":"2024-05-03T20:33:42.714533+0000","last_undegraded":"2024-05-03T20:33:42.714533+0000","last_fullsized":"2024-05-03T20:33:42.714533+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:28:56.388292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:58.493919+0000","last_change":"2024-05-03T20:33:37.683740+0000","last_active":"2024-05-03T20:33:58.493919+0000","last_peered":"2024-05-03T20:33:58.493919+0000","last_clean":"2024-05-03T20:33:58.493919+0000","last_became_active":"2024-05-03T20:33:37.683046+0000","last_became_peered":"2024-05-03T20:33:37.683046+0000","last_unstale":"2024-05-03T20:33:58.493919+0000","last_undegraded":"2024-05-03T20:33:58.493919+0000","last_fullsized":"2024-05-03T20:33:58.493919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:41:58.066741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.712015+0000","last_change":"2024-05-03T20:33:40.711274+0000","last_active":"2024-05-03T20:33:42.712015+0000","last_peered":"2024-05-03T20:33:42.712015+0000","last_clean":"2024-05-03T20:33:42.712015+0000","last_became_active":"2024-05-03T20:33:40.710917+0000","last_became_peered":"2024-05-03T20:33:40.710917+0000","last_unstale":"2024-05-03T20:33:42.712015+0000","last_undegraded":"2024-05-03T20:33:42.712015+0000","last_fullsized":"2024-05-03T20:33:42.712015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:22:55.969325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:57.485975+0000","last_change":"2024-05-03T20:33:37.681383+0000","last_active":"2024-05-03T20:33:57.485975+0000","last_peered":"2024-05-03T20:33:57.485975+0000","last_clean":"2024-05-03T20:33:57.485975+0000","last_became_active":"2024-05-03T20:33:37.680941+0000","last_became_peered":"2024-05-03T20:33:37.680941+0000","last_unstale":"2024-05-03T20:33:57.485975+0000","last_undegraded":"2024-05-03T20:33:57.485975+0000","last_fullsized":"2024-05-03T20:33:57.485975+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:03:26.083579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.712145+0000","last_change":"2024-05-03T20:33:40.711762+0000","last_active":"2024-05-03T20:33:42.712145+0000","last_peered":"2024-05-03T20:33:42.712145+0000","last_clean":"2024-05-03T20:33:42.712145+0000","last_became_active":"2024-05-03T20:33:40.710630+0000","last_became_peered":"2024-05-03T20:33:40.710630+0000","last_unstale":"2024-05-03T20:33:42.712145+0000","last_undegraded":"2024-05-03T20:33:42.712145+0000","last_fullsized":"2024-05-03T20:33:42.712145+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:42:23.728916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":54,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:34:20.524698+0000","last_change":"2024-05-03T20:33:37.685895+0000","last_active":"2024-05-03T20:34:20.524698+0000","last_peered":"2024-05-03T20:34:20.524698+0000","last_clean":"2024-05-03T20:34:20.524698+0000","last_became_active":"2024-05-03T20:33:37.685399+0000","last_became_peered":"2024-05-03T20:33:37.685399+0000","last_unstale":"2024-05-03T20:34:20.524698+0000","last_undegraded":"2024-05-03T20:34:20.524698+0000","last_fullsized":"2024-05-03T20:34:20.524698+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:09:29.932599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711262+0000","last_change":"2024-05-03T20:33:40.704967+0000","last_active":"2024-05-03T20:33:42.711262+0000","last_peered":"2024-05-03T20:33:42.711262+0000","last_clean":"2024-05-03T20:33:42.711262+0000","last_became_active":"2024-05-03T20:33:40.704583+0000","last_became_peered":"2024-05-03T20:33:40.704583+0000","last_unstale":"2024-05-03T20:33:42.711262+0000","last_undegraded":"2024-05-03T20:33:42.711262+0000","last_fullsized":"2024-05-03T20:33:42.711262+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:46:31.720865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:56.475217+0000","last_change":"2024-05-03T20:33:37.680798+0000","last_active":"2024-05-03T20:33:56.475217+0000","last_peered":"2024-05-03T20:33:56.475217+0000","last_clean":"2024-05-03T20:33:56.475217+0000","last_became_active":"2024-05-03T20:33:37.680344+0000","last_became_peered":"2024-05-03T20:33:37.680344+0000","last_unstale":"2024-05-03T20:33:56.475217+0000","last_undegraded":"2024-05-03T20:33:56.475217+0000","last_fullsized":"2024-05-03T20:33:56.475217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:26:32.037104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.712254+0000","last_change":"2024-05-03T20:33:40.710976+0000","last_active":"2024-05-03T20:33:42.712254+0000","last_peered":"2024-05-03T20:33:42.712254+0000","last_clean":"2024-05-03T20:33:42.712254+0000","last_became_active":"2024-05-03T20:33:40.710649+0000","last_became_peered":"2024-05-03T20:33:40.710649+0000","last_unstale":"2024-05-03T20:33:42.712254+0000","last_undegraded":"2024-05-03T20:33:42.712254+0000","last_fullsized":"2024-05-03T20:33:42.712254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:57:30.367412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:55.467898+0000","last_change":"2024-05-03T20:33:37.684230+0000","last_active":"2024-05-03T20:33:55.467898+0000","last_peered":"2024-05-03T20:33:55.467898+0000","last_clean":"2024-05-03T20:33:55.467898+0000","last_became_active":"2024-05-03T20:33:37.683844+0000","last_became_peered":"2024-05-03T20:33:37.683844+0000","last_unstale":"2024-05-03T20:33:55.467898+0000","last_undegraded":"2024-05-03T20:33:55.467898+0000","last_fullsized":"2024-05-03T20:33:55.467898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:46:23.417163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711997+0000","last_change":"2024-05-03T20:33:40.706554+0000","last_active":"2024-05-03T20:33:42.711997+0000","last_peered":"2024-05-03T20:33:42.711997+0000","last_clean":"2024-05-03T20:33:42.711997+0000","last_became_active":"2024-05-03T20:33:40.706142+0000","last_became_peered":"2024-05-03T20:33:40.706142+0000","last_unstale":"2024-05-03T20:33:42.711997+0000","last_undegraded":"2024-05-03T20:33:42.711997+0000","last_fullsized":"2024-05-03T20:33:42.711997+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:01:28.324024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:57.484669+0000","last_change":"2024-05-03T20:33:37.683497+0000","last_active":"2024-05-03T20:33:57.484669+0000","last_peered":"2024-05-03T20:33:57.484669+0000","last_clean":"2024-05-03T20:33:57.484669+0000","last_became_active":"2024-05-03T20:33:37.683178+0000","last_became_peered":"2024-05-03T20:33:37.683178+0000","last_unstale":"2024-05-03T20:33:57.484669+0000","last_undegraded":"2024-05-03T20:33:57.484669+0000","last_fullsized":"2024-05-03T20:33:57.484669+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:07:29.771786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711424+0000","last_change":"2024-05-03T20:33:40.705447+0000","last_active":"2024-05-03T20:33:42.711424+0000","last_peered":"2024-05-03T20:33:42.711424+0000","last_clean":"2024-05-03T20:33:42.711424+0000","last_became_active":"2024-05-03T20:33:40.705102+0000","last_became_peered":"2024-05-03T20:33:40.705102+0000","last_unstale":"2024-05-03T20:33:42.711424+0000","last_undegraded":"2024-05-03T20:33:42.711424+0000","last_fullsized":"2024-05-03T20:33:42.711424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:55:55.354498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":75,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:34:22.526873+0000","last_change":"2024-05-03T20:33:37.681449+0000","last_active":"2024-05-03T20:34:22.526873+0000","last_peered":"2024-05-03T20:34:22.526873+0000","last_clean":"2024-05-03T20:34:22.526873+0000","last_became_active":"2024-05-03T20:33:37.680798+0000","last_became_peered":"2024-05-03T20:33:37.680798+0000","last_unstale":"2024-05-03T20:34:22.526873+0000","last_undegraded":"2024-05-03T20:34:22.526873+0000","last_fullsized":"2024-05-03T20:34:22.526873+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:42:10.029261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.712325+0000","last_change":"2024-05-03T20:33:40.708872+0000","last_active":"2024-05-03T20:33:42.712325+0000","last_peered":"2024-05-03T20:33:42.712325+0000","last_clean":"2024-05-03T20:33:42.712325+0000","last_became_active":"2024-05-03T20:33:40.708535+0000","last_became_peered":"2024-05-03T20:33:40.708535+0000","last_unstale":"2024-05-03T20:33:42.712325+0000","last_undegraded":"2024-05-03T20:33:42.712325+0000","last_fullsized":"2024-05-03T20:33:42.712325+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:21:08.949065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:57.484290+0000","last_change":"2024-05-03T20:33:37.685425+0000","last_active":"2024-05-03T20:33:57.484290+0000","last_peered":"2024-05-03T20:33:57.484290+0000","last_clean":"2024-05-03T20:33:57.484290+0000","last_became_active":"2024-05-03T20:33:37.685032+0000","last_became_peered":"2024-05-03T20:33:37.685032+0000","last_unstale":"2024-05-03T20:33:57.484290+0000","last_undegraded":"2024-05-03T20:33:57.484290+0000","last_fullsized":"2024-05-03T20:33:57.484290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:14:35.380780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714435+0000","last_change":"2024-05-03T20:33:40.713991+0000","last_active":"2024-05-03T20:33:42.714435+0000","last_peered":"2024-05-03T20:33:42.714435+0000","last_clean":"2024-05-03T20:33:42.714435+0000","last_became_active":"2024-05-03T20:33:40.713661+0000","last_became_peered":"2024-05-03T20:33:40.713661+0000","last_unstale":"2024-05-03T20:33:42.714435+0000","last_undegraded":"2024-05-03T20:33:42.714435+0000","last_fullsized":"2024-05-03T20:33:42.714435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:08:20.818863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:56.474686+0000","last_change":"2024-05-03T20:33:38.237724+0000","last_active":"2024-05-03T20:33:56.474686+0000","last_peered":"2024-05-03T20:33:56.474686+0000","last_clean":"2024-05-03T20:33:56.474686+0000","last_became_active":"2024-05-03T20:33:38.237340+0000","last_became_peered":"2024-05-03T20:33:38.237340+0000","last_unstale":"2024-05-03T20:33:56.474686+0000","last_undegraded":"2024-05-03T20:33:56.474686+0000","last_fullsized":"2024-05-03T20:33:56.474686+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:13:35.567527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.712380+0000","last_change":"2024-05-03T20:33:40.707454+0000","last_active":"2024-05-03T20:33:42.712380+0000","last_peered":"2024-05-03T20:33:42.712380+0000","last_clean":"2024-05-03T20:33:42.712380+0000","last_became_active":"2024-05-03T20:33:40.707080+0000","last_became_peered":"2024-05-03T20:33:40.707080+0000","last_unstale":"2024-05-03T20:33:42.712380+0000","last_undegraded":"2024-05-03T20:33:42.712380+0000","last_fullsized":"2024-05-03T20:33:42.712380+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:12:18.432048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'27","reported_seq":105,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:34:22.527878+0000","last_change":"2024-05-03T20:33:37.681144+0000","last_active":"2024-05-03T20:34:22.527878+0000","last_peered":"2024-05-03T20:34:22.527878+0000","last_clean":"2024-05-03T20:34:22.527878+0000","last_became_active":"2024-05-03T20:33:37.680531+0000","last_became_peered":"2024-05-03T20:33:37.680531+0000","last_unstale":"2024-05-03T20:34:22.527878+0000","last_undegraded":"2024-05-03T20:34:22.527878+0000","last_fullsized":"2024-05-03T20:34:22.527878+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:45:53.536622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49601,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":57,"num_read_kb":76,"num_write":29,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711916+0000","last_change":"2024-05-03T20:33:40.711336+0000","last_active":"2024-05-03T20:33:42.711916+0000","last_peered":"2024-05-03T20:33:42.711916+0000","last_clean":"2024-05-03T20:33:42.711916+0000","last_became_active":"2024-05-03T20:33:40.710991+0000","last_became_peered":"2024-05-03T20:33:40.710991+0000","last_unstale":"2024-05-03T20:33:42.711916+0000","last_undegraded":"2024-05-03T20:33:42.711916+0000","last_fullsized":"2024-05-03T20:33:42.711916+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:59:44.729974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:58.493166+0000","last_change":"2024-05-03T20:33:37.682280+0000","last_active":"2024-05-03T20:33:58.493166+0000","last_peered":"2024-05-03T20:33:58.493166+0000","last_clean":"2024-05-03T20:33:58.493166+0000","last_became_active":"2024-05-03T20:33:37.681612+0000","last_became_peered":"2024-05-03T20:33:37.681612+0000","last_unstale":"2024-05-03T20:33:58.493166+0000","last_undegraded":"2024-05-03T20:33:58.493166+0000","last_fullsized":"2024-05-03T20:33:58.493166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:50:05.310560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.698280+0000","last_change":"2024-05-03T20:33:40.710309+0000","last_active":"2024-05-03T20:33:42.698280+0000","last_peered":"2024-05-03T20:33:42.698280+0000","last_clean":"2024-05-03T20:33:42.698280+0000","last_became_active":"2024-05-03T20:33:40.710015+0000","last_became_peered":"2024-05-03T20:33:40.710015+0000","last_unstale":"2024-05-03T20:33:42.698280+0000","last_undegraded":"2024-05-03T20:33:42.698280+0000","last_fullsized":"2024-05-03T20:33:42.698280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:05:04.042635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:57.484730+0000","last_change":"2024-05-03T20:33:37.679525+0000","last_active":"2024-05-03T20:33:57.484730+0000","last_peered":"2024-05-03T20:33:57.484730+0000","last_clean":"2024-05-03T20:33:57.484730+0000","last_became_active":"2024-05-03T20:33:37.678905+0000","last_became_peered":"2024-05-03T20:33:37.678905+0000","last_unstale":"2024-05-03T20:33:57.484730+0000","last_undegraded":"2024-05-03T20:33:57.484730+0000","last_fullsized":"2024-05-03T20:33:57.484730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:17:06.225255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714612+0000","last_change":"2024-05-03T20:33:40.717613+0000","last_active":"2024-05-03T20:33:42.714612+0000","last_peered":"2024-05-03T20:33:42.714612+0000","last_clean":"2024-05-03T20:33:42.714612+0000","last_became_active":"2024-05-03T20:33:40.717348+0000","last_became_peered":"2024-05-03T20:33:40.717348+0000","last_unstale":"2024-05-03T20:33:42.714612+0000","last_undegraded":"2024-05-03T20:33:42.714612+0000","last_fullsized":"2024-05-03T20:33:42.714612+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:20:31.493731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'6","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:34:20.524037+0000","last_change":"2024-05-03T20:33:37.683083+0000","last_active":"2024-05-03T20:34:20.524037+0000","last_peered":"2024-05-03T20:34:20.524037+0000","last_clean":"2024-05-03T20:34:20.524037+0000","last_became_active":"2024-05-03T20:33:37.682700+0000","last_became_peered":"2024-05-03T20:33:37.682700+0000","last_unstale":"2024-05-03T20:34:20.524037+0000","last_undegraded":"2024-05-03T20:34:20.524037+0000","last_fullsized":"2024-05-03T20:34:20.524037+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:47:18.294314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714220+0000","last_change":"2024-05-03T20:33:40.711711+0000","last_active":"2024-05-03T20:33:42.714220+0000","last_peered":"2024-05-03T20:33:42.714220+0000","last_clean":"2024-05-03T20:33:42.714220+0000","last_became_active":"2024-05-03T20:33:40.711116+0000","last_became_peered":"2024-05-03T20:33:40.711116+0000","last_unstale":"2024-05-03T20:33:42.714220+0000","last_undegraded":"2024-05-03T20:33:42.714220+0000","last_fullsized":"2024-05-03T20:33:42.714220+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:53:18.231575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:57.485691+0000","last_change":"2024-05-03T20:33:38.238005+0000","last_active":"2024-05-03T20:33:57.485691+0000","last_peered":"2024-05-03T20:33:57.485691+0000","last_clean":"2024-05-03T20:33:57.485691+0000","last_became_active":"2024-05-03T20:33:38.237636+0000","last_became_peered":"2024-05-03T20:33:38.237636+0000","last_unstale":"2024-05-03T20:33:57.485691+0000","last_undegraded":"2024-05-03T20:33:57.485691+0000","last_fullsized":"2024-05-03T20:33:57.485691+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:49:15.501715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711951+0000","last_change":"2024-05-03T20:33:40.716317+0000","last_active":"2024-05-03T20:33:42.711951+0000","last_peered":"2024-05-03T20:33:42.711951+0000","last_clean":"2024-05-03T20:33:42.711951+0000","last_became_active":"2024-05-03T20:33:40.715780+0000","last_became_peered":"2024-05-03T20:33:40.715780+0000","last_unstale":"2024-05-03T20:33:42.711951+0000","last_undegraded":"2024-05-03T20:33:42.711951+0000","last_fullsized":"2024-05-03T20:33:42.711951+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:21:46.186992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:56.462813+0000","last_change":"2024-05-03T20:33:37.679690+0000","last_active":"2024-05-03T20:33:56.462813+0000","last_peered":"2024-05-03T20:33:56.462813+0000","last_clean":"2024-05-03T20:33:56.462813+0000","last_became_active":"2024-05-03T20:33:37.679214+0000","last_became_peered":"2024-05-03T20:33:37.679214+0000","last_unstale":"2024-05-03T20:33:56.462813+0000","last_undegraded":"2024-05-03T20:33:56.462813+0000","last_fullsized":"2024-05-03T20:33:56.462813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:08:54.467850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.699036+0000","last_change":"2024-05-03T20:33:40.714800+0000","last_active":"2024-05-03T20:33:42.699036+0000","last_peered":"2024-05-03T20:33:42.699036+0000","last_clean":"2024-05-03T20:33:42.699036+0000","last_became_active":"2024-05-03T20:33:40.714417+0000","last_became_peered":"2024-05-03T20:33:40.714417+0000","last_unstale":"2024-05-03T20:33:42.699036+0000","last_undegraded":"2024-05-03T20:33:42.699036+0000","last_fullsized":"2024-05-03T20:33:42.699036+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:06:37.110378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.710999+0000","last_change":"2024-05-03T20:33:38.237270+0000","last_active":"2024-05-03T20:33:42.710999+0000","last_peered":"2024-05-03T20:33:42.710999+0000","last_clean":"2024-05-03T20:33:42.710999+0000","last_became_active":"2024-05-03T20:33:38.236917+0000","last_became_peered":"2024-05-03T20:33:38.236917+0000","last_unstale":"2024-05-03T20:33:42.710999+0000","last_undegraded":"2024-05-03T20:33:42.710999+0000","last_fullsized":"2024-05-03T20:33:42.710999+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:38:58.079905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:43.026647+0000","last_change":"2024-05-03T20:33:40.707368+0000","last_active":"2024-05-03T20:33:43.026647+0000","last_peered":"2024-05-03T20:33:43.026647+0000","last_clean":"2024-05-03T20:33:43.026647+0000","last_became_active":"2024-05-03T20:33:40.707049+0000","last_became_peered":"2024-05-03T20:33:40.707049+0000","last_unstale":"2024-05-03T20:33:43.026647+0000","last_undegraded":"2024-05-03T20:33:43.026647+0000","last_fullsized":"2024-05-03T20:33:43.026647+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:02:57.790142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'6","reported_seq":52,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:34:18.521304+0000","last_change":"2024-05-03T20:33:37.679007+0000","last_active":"2024-05-03T20:34:18.521304+0000","last_peered":"2024-05-03T20:34:18.521304+0000","last_clean":"2024-05-03T20:34:18.521304+0000","last_became_active":"2024-05-03T20:33:37.678609+0000","last_became_peered":"2024-05-03T20:33:37.678609+0000","last_unstale":"2024-05-03T20:34:18.521304+0000","last_undegraded":"2024-05-03T20:34:18.521304+0000","last_fullsized":"2024-05-03T20:34:18.521304+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:30:17.622905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.696804+0000","last_change":"2024-05-03T20:33:40.713820+0000","last_active":"2024-05-03T20:33:42.696804+0000","last_peered":"2024-05-03T20:33:42.696804+0000","last_clean":"2024-05-03T20:33:42.696804+0000","last_became_active":"2024-05-03T20:33:40.713487+0000","last_became_peered":"2024-05-03T20:33:40.713487+0000","last_unstale":"2024-05-03T20:33:42.696804+0000","last_undegraded":"2024-05-03T20:33:42.696804+0000","last_fullsized":"2024-05-03T20:33:42.696804+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:23:03.724180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714153+0000","last_change":"2024-05-03T20:33:40.716663+0000","last_active":"2024-05-03T20:33:42.714153+0000","last_peered":"2024-05-03T20:33:42.714153+0000","last_clean":"2024-05-03T20:33:42.714153+0000","last_became_active":"2024-05-03T20:33:40.716201+0000","last_became_peered":"2024-05-03T20:33:40.716201+0000","last_unstale":"2024-05-03T20:33:42.714153+0000","last_undegraded":"2024-05-03T20:33:42.714153+0000","last_fullsized":"2024-05-03T20:33:42.714153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:38:12.975351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:34:20.523430+0000","last_change":"2024-05-03T20:33:38.237285+0000","last_active":"2024-05-03T20:34:20.523430+0000","last_peered":"2024-05-03T20:34:20.523430+0000","last_clean":"2024-05-03T20:34:20.523430+0000","last_became_active":"2024-05-03T20:33:38.236825+0000","last_became_peered":"2024-05-03T20:33:38.236825+0000","last_unstale":"2024-05-03T20:34:20.523430+0000","last_undegraded":"2024-05-03T20:34:20.523430+0000","last_fullsized":"2024-05-03T20:34:20.523430+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:46:52.963953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.712414+0000","last_change":"2024-05-03T20:33:40.713271+0000","last_active":"2024-05-03T20:33:42.712414+0000","last_peered":"2024-05-03T20:33:42.712414+0000","last_clean":"2024-05-03T20:33:42.712414+0000","last_became_active":"2024-05-03T20:33:40.712972+0000","last_became_peered":"2024-05-03T20:33:40.712972+0000","last_unstale":"2024-05-03T20:33:42.712414+0000","last_undegraded":"2024-05-03T20:33:42.712414+0000","last_fullsized":"2024-05-03T20:33:42.712414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:39:46.762096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:58.493230+0000","last_change":"2024-05-03T20:33:37.682196+0000","last_active":"2024-05-03T20:33:58.493230+0000","last_peered":"2024-05-03T20:33:58.493230+0000","last_clean":"2024-05-03T20:33:58.493230+0000","last_became_active":"2024-05-03T20:33:37.681671+0000","last_became_peered":"2024-05-03T20:33:37.681671+0000","last_unstale":"2024-05-03T20:33:58.493230+0000","last_undegraded":"2024-05-03T20:33:58.493230+0000","last_fullsized":"2024-05-03T20:33:58.493230+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:53:26.866288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":97,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714588+0000","last_change":"2024-05-03T20:32:15.372604+0000","last_active":"2024-05-03T20:33:42.714588+0000","last_peered":"2024-05-03T20:33:42.714588+0000","last_clean":"2024-05-03T20:33:42.714588+0000","last_became_active":"2024-05-03T20:32:15.366768+0000","last_became_peered":"2024-05-03T20:32:15.366768+0000","last_unstale":"2024-05-03T20:33:42.714588+0000","last_undegraded":"2024-05-03T20:33:42.714588+0000","last_fullsized":"2024-05-03T20:33:42.714588+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:29:02.045902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:29:02.045902+0000","last_clean_scrub_stamp":"2024-05-03T20:29:02.045902+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:02:05.424270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.698260+0000","last_change":"2024-05-03T20:33:40.710085+0000","last_active":"2024-05-03T20:33:42.698260+0000","last_peered":"2024-05-03T20:33:42.698260+0000","last_clean":"2024-05-03T20:33:42.698260+0000","last_became_active":"2024-05-03T20:33:40.709765+0000","last_became_peered":"2024-05-03T20:33:40.709765+0000","last_unstale":"2024-05-03T20:33:42.698260+0000","last_undegraded":"2024-05-03T20:33:42.698260+0000","last_fullsized":"2024-05-03T20:33:42.698260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:52:02.339529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:55.467593+0000","last_change":"2024-05-03T20:33:37.679697+0000","last_active":"2024-05-03T20:33:55.467593+0000","last_peered":"2024-05-03T20:33:55.467593+0000","last_clean":"2024-05-03T20:33:55.467593+0000","last_became_active":"2024-05-03T20:33:37.679301+0000","last_became_peered":"2024-05-03T20:33:37.679301+0000","last_unstale":"2024-05-03T20:33:55.467593+0000","last_undegraded":"2024-05-03T20:33:55.467593+0000","last_fullsized":"2024-05-03T20:33:55.467593+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:17:11.201482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:57.486253+0000","last_change":"2024-05-03T20:33:38.239293+0000","last_active":"2024-05-03T20:33:57.486253+0000","last_peered":"2024-05-03T20:33:57.486253+0000","last_clean":"2024-05-03T20:33:57.486253+0000","last_became_active":"2024-05-03T20:33:38.238888+0000","last_became_peered":"2024-05-03T20:33:38.238888+0000","last_unstale":"2024-05-03T20:33:57.486253+0000","last_undegraded":"2024-05-03T20:33:57.486253+0000","last_fullsized":"2024-05-03T20:33:57.486253+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:41:06.397280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.698576+0000","last_change":"2024-05-03T20:33:40.710180+0000","last_active":"2024-05-03T20:33:42.698576+0000","last_peered":"2024-05-03T20:33:42.698576+0000","last_clean":"2024-05-03T20:33:42.698576+0000","last_became_active":"2024-05-03T20:33:40.709322+0000","last_became_peered":"2024-05-03T20:33:40.709322+0000","last_unstale":"2024-05-03T20:33:42.698576+0000","last_undegraded":"2024-05-03T20:33:42.698576+0000","last_fullsized":"2024-05-03T20:33:42.698576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:15:44.569110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'13","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:34:13.713611+0000","last_change":"2024-05-03T20:33:37.682422+0000","last_active":"2024-05-03T20:34:13.713611+0000","last_peered":"2024-05-03T20:34:13.713611+0000","last_clean":"2024-05-03T20:34:13.713611+0000","last_became_active":"2024-05-03T20:33:37.682079+0000","last_became_peered":"2024-05-03T20:33:37.682079+0000","last_unstale":"2024-05-03T20:34:13.713611+0000","last_undegraded":"2024-05-03T20:34:13.713611+0000","last_fullsized":"2024-05-03T20:34:13.713611+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:54:43.806768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711943+0000","last_change":"2024-05-03T20:33:40.715859+0000","last_active":"2024-05-03T20:33:42.711943+0000","last_peered":"2024-05-03T20:33:42.711943+0000","last_clean":"2024-05-03T20:33:42.711943+0000","last_became_active":"2024-05-03T20:33:40.715511+0000","last_became_peered":"2024-05-03T20:33:40.715511+0000","last_unstale":"2024-05-03T20:33:42.711943+0000","last_undegraded":"2024-05-03T20:33:42.711943+0000","last_fullsized":"2024-05-03T20:33:42.711943+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:27:59.317403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:57.485633+0000","last_change":"2024-05-03T20:33:37.679161+0000","last_active":"2024-05-03T20:33:57.485633+0000","last_peered":"2024-05-03T20:33:57.485633+0000","last_clean":"2024-05-03T20:33:57.485633+0000","last_became_active":"2024-05-03T20:33:37.678825+0000","last_became_peered":"2024-05-03T20:33:37.678825+0000","last_unstale":"2024-05-03T20:33:57.485633+0000","last_undegraded":"2024-05-03T20:33:57.485633+0000","last_fullsized":"2024-05-03T20:33:57.485633+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:05:01.203880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714331+0000","last_change":"2024-05-03T20:33:40.713287+0000","last_active":"2024-05-03T20:33:42.714331+0000","last_peered":"2024-05-03T20:33:42.714331+0000","last_clean":"2024-05-03T20:33:42.714331+0000","last_became_active":"2024-05-03T20:33:40.712962+0000","last_became_peered":"2024-05-03T20:33:40.712962+0000","last_unstale":"2024-05-03T20:33:42.714331+0000","last_undegraded":"2024-05-03T20:33:42.714331+0000","last_fullsized":"2024-05-03T20:33:42.714331+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:22:59.151645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:54.461104+0000","last_change":"2024-05-03T20:33:38.237996+0000","last_active":"2024-05-03T20:33:54.461104+0000","last_peered":"2024-05-03T20:33:54.461104+0000","last_clean":"2024-05-03T20:33:54.461104+0000","last_became_active":"2024-05-03T20:33:38.237552+0000","last_became_peered":"2024-05-03T20:33:38.237552+0000","last_unstale":"2024-05-03T20:33:54.461104+0000","last_undegraded":"2024-05-03T20:33:54.461104+0000","last_fullsized":"2024-05-03T20:33:54.461104+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:23:47.946259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711154+0000","last_change":"2024-05-03T20:33:40.709634+0000","last_active":"2024-05-03T20:33:42.711154+0000","last_peered":"2024-05-03T20:33:42.711154+0000","last_clean":"2024-05-03T20:33:42.711154+0000","last_became_active":"2024-05-03T20:33:40.709195+0000","last_became_peered":"2024-05-03T20:33:40.709195+0000","last_unstale":"2024-05-03T20:33:42.711154+0000","last_undegraded":"2024-05-03T20:33:42.711154+0000","last_fullsized":"2024-05-03T20:33:42.711154+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:24:30.796765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:57.484732+0000","last_change":"2024-05-03T20:33:37.677729+0000","last_active":"2024-05-03T20:33:57.484732+0000","last_peered":"2024-05-03T20:33:57.484732+0000","last_clean":"2024-05-03T20:33:57.484732+0000","last_became_active":"2024-05-03T20:33:37.677285+0000","last_became_peered":"2024-05-03T20:33:37.677285+0000","last_unstale":"2024-05-03T20:33:57.484732+0000","last_undegraded":"2024-05-03T20:33:57.484732+0000","last_fullsized":"2024-05-03T20:33:57.484732+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:01:46.289588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.698655+0000","last_change":"2024-05-03T20:33:40.708573+0000","last_active":"2024-05-03T20:33:42.698655+0000","last_peered":"2024-05-03T20:33:42.698655+0000","last_clean":"2024-05-03T20:33:42.698655+0000","last_became_active":"2024-05-03T20:33:40.708238+0000","last_became_peered":"2024-05-03T20:33:40.708238+0000","last_unstale":"2024-05-03T20:33:42.698655+0000","last_undegraded":"2024-05-03T20:33:42.698655+0000","last_fullsized":"2024-05-03T20:33:42.698655+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:21:06.975075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:56.475851+0000","last_change":"2024-05-03T20:33:37.683415+0000","last_active":"2024-05-03T20:33:56.475851+0000","last_peered":"2024-05-03T20:33:56.475851+0000","last_clean":"2024-05-03T20:33:56.475851+0000","last_became_active":"2024-05-03T20:33:37.682935+0000","last_became_peered":"2024-05-03T20:33:37.682935+0000","last_unstale":"2024-05-03T20:33:56.475851+0000","last_undegraded":"2024-05-03T20:33:56.475851+0000","last_fullsized":"2024-05-03T20:33:56.475851+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:26:17.977777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711994+0000","last_change":"2024-05-03T20:33:40.708116+0000","last_active":"2024-05-03T20:33:42.711994+0000","last_peered":"2024-05-03T20:33:42.711994+0000","last_clean":"2024-05-03T20:33:42.711994+0000","last_became_active":"2024-05-03T20:33:40.707625+0000","last_became_peered":"2024-05-03T20:33:40.707625+0000","last_unstale":"2024-05-03T20:33:42.711994+0000","last_undegraded":"2024-05-03T20:33:42.711994+0000","last_fullsized":"2024-05-03T20:33:42.711994+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:52:44.984252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:54.462116+0000","last_change":"2024-05-03T20:33:37.685132+0000","last_active":"2024-05-03T20:33:54.462116+0000","last_peered":"2024-05-03T20:33:54.462116+0000","last_clean":"2024-05-03T20:33:54.462116+0000","last_became_active":"2024-05-03T20:33:37.684169+0000","last_became_peered":"2024-05-03T20:33:37.684169+0000","last_unstale":"2024-05-03T20:33:54.462116+0000","last_undegraded":"2024-05-03T20:33:54.462116+0000","last_fullsized":"2024-05-03T20:33:54.462116+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:41:24.163383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.715212+0000","last_change":"2024-05-03T20:33:40.713622+0000","last_active":"2024-05-03T20:33:42.715212+0000","last_peered":"2024-05-03T20:33:42.715212+0000","last_clean":"2024-05-03T20:33:42.715212+0000","last_became_active":"2024-05-03T20:33:40.712467+0000","last_became_peered":"2024-05-03T20:33:40.712467+0000","last_unstale":"2024-05-03T20:33:42.715212+0000","last_undegraded":"2024-05-03T20:33:42.715212+0000","last_fullsized":"2024-05-03T20:33:42.715212+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:40:07.948287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'5","reported_seq":52,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:34:19.523277+0000","last_change":"2024-05-03T20:33:38.237790+0000","last_active":"2024-05-03T20:34:19.523277+0000","last_peered":"2024-05-03T20:34:19.523277+0000","last_clean":"2024-05-03T20:34:19.523277+0000","last_became_active":"2024-05-03T20:33:38.237327+0000","last_became_peered":"2024-05-03T20:33:38.237327+0000","last_unstale":"2024-05-03T20:34:19.523277+0000","last_undegraded":"2024-05-03T20:34:19.523277+0000","last_fullsized":"2024-05-03T20:34:19.523277+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:59:15.465519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":25,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.715061+0000","last_change":"2024-05-03T20:33:40.710127+0000","last_active":"2024-05-03T20:33:42.715061+0000","last_peered":"2024-05-03T20:33:42.715061+0000","last_clean":"2024-05-03T20:33:42.715061+0000","last_became_active":"2024-05-03T20:33:40.709743+0000","last_became_peered":"2024-05-03T20:33:40.709743+0000","last_unstale":"2024-05-03T20:33:42.715061+0000","last_undegraded":"2024-05-03T20:33:42.715061+0000","last_fullsized":"2024-05-03T20:33:42.715061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:01:48.638259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:57.484711+0000","last_change":"2024-05-03T20:33:37.680009+0000","last_active":"2024-05-03T20:33:57.484711+0000","last_peered":"2024-05-03T20:33:57.484711+0000","last_clean":"2024-05-03T20:33:57.484711+0000","last_became_active":"2024-05-03T20:33:37.679577+0000","last_became_peered":"2024-05-03T20:33:37.679577+0000","last_unstale":"2024-05-03T20:33:57.484711+0000","last_undegraded":"2024-05-03T20:33:57.484711+0000","last_fullsized":"2024-05-03T20:33:57.484711+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:13:01.801473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714845+0000","last_change":"2024-05-03T20:33:40.714713+0000","last_active":"2024-05-03T20:33:42.714845+0000","last_peered":"2024-05-03T20:33:42.714845+0000","last_clean":"2024-05-03T20:33:42.714845+0000","last_became_active":"2024-05-03T20:33:40.711743+0000","last_became_peered":"2024-05-03T20:33:40.711743+0000","last_unstale":"2024-05-03T20:33:42.714845+0000","last_undegraded":"2024-05-03T20:33:42.714845+0000","last_fullsized":"2024-05-03T20:33:42.714845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:57:42.748953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:57.484323+0000","last_change":"2024-05-03T20:33:37.682038+0000","last_active":"2024-05-03T20:33:57.484323+0000","last_peered":"2024-05-03T20:33:57.484323+0000","last_clean":"2024-05-03T20:33:57.484323+0000","last_became_active":"2024-05-03T20:33:37.681559+0000","last_became_peered":"2024-05-03T20:33:37.681559+0000","last_unstale":"2024-05-03T20:33:57.484323+0000","last_undegraded":"2024-05-03T20:33:57.484323+0000","last_fullsized":"2024-05-03T20:33:57.484323+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:12:49.532764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.715015+0000","last_change":"2024-05-03T20:33:40.708025+0000","last_active":"2024-05-03T20:33:42.715015+0000","last_peered":"2024-05-03T20:33:42.715015+0000","last_clean":"2024-05-03T20:33:42.715015+0000","last_became_active":"2024-05-03T20:33:40.707601+0000","last_became_peered":"2024-05-03T20:33:40.707601+0000","last_unstale":"2024-05-03T20:33:42.715015+0000","last_undegraded":"2024-05-03T20:33:42.715015+0000","last_fullsized":"2024-05-03T20:33:42.715015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:34:32.060247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.698962+0000","last_change":"2024-05-03T20:33:40.711728+0000","last_active":"2024-05-03T20:33:42.698962+0000","last_peered":"2024-05-03T20:33:42.698962+0000","last_clean":"2024-05-03T20:33:42.698962+0000","last_became_active":"2024-05-03T20:33:40.711367+0000","last_became_peered":"2024-05-03T20:33:40.711367+0000","last_unstale":"2024-05-03T20:33:42.698962+0000","last_undegraded":"2024-05-03T20:33:42.698962+0000","last_fullsized":"2024-05-03T20:33:42.698962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:07:17.956339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.713753+0000","last_change":"2024-05-03T20:33:40.716758+0000","last_active":"2024-05-03T20:33:42.713753+0000","last_peered":"2024-05-03T20:33:42.713753+0000","last_clean":"2024-05-03T20:33:42.713753+0000","last_became_active":"2024-05-03T20:33:40.716281+0000","last_became_peered":"2024-05-03T20:33:40.716281+0000","last_unstale":"2024-05-03T20:33:42.713753+0000","last_undegraded":"2024-05-03T20:33:42.713753+0000","last_fullsized":"2024-05-03T20:33:42.713753+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:59:40.795475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.712508+0000","last_change":"2024-05-03T20:33:40.705468+0000","last_active":"2024-05-03T20:33:42.712508+0000","last_peered":"2024-05-03T20:33:42.712508+0000","last_clean":"2024-05-03T20:33:42.712508+0000","last_became_active":"2024-05-03T20:33:40.705114+0000","last_became_peered":"2024-05-03T20:33:40.705114+0000","last_unstale":"2024-05-03T20:33:42.712508+0000","last_undegraded":"2024-05-03T20:33:42.712508+0000","last_fullsized":"2024-05-03T20:33:42.712508+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:20:05.549638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.714387+0000","last_change":"2024-05-03T20:33:40.713282+0000","last_active":"2024-05-03T20:33:42.714387+0000","last_peered":"2024-05-03T20:33:42.714387+0000","last_clean":"2024-05-03T20:33:42.714387+0000","last_became_active":"2024-05-03T20:33:40.712674+0000","last_became_peered":"2024-05-03T20:33:40.712674+0000","last_unstale":"2024-05-03T20:33:42.714387+0000","last_undegraded":"2024-05-03T20:33:42.714387+0000","last_fullsized":"2024-05-03T20:33:42.714387+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:46:53.034439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.698677+0000","last_change":"2024-05-03T20:33:40.707870+0000","last_active":"2024-05-03T20:33:42.698677+0000","last_peered":"2024-05-03T20:33:42.698677+0000","last_clean":"2024-05-03T20:33:42.698677+0000","last_became_active":"2024-05-03T20:33:40.706849+0000","last_became_peered":"2024-05-03T20:33:40.706849+0000","last_unstale":"2024-05-03T20:33:42.698677+0000","last_undegraded":"2024-05-03T20:33:42.698677+0000","last_fullsized":"2024-05-03T20:33:42.698677+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:23:18.030508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.699007+0000","last_change":"2024-05-03T20:33:40.716365+0000","last_active":"2024-05-03T20:33:42.699007+0000","last_peered":"2024-05-03T20:33:42.699007+0000","last_clean":"2024-05-03T20:33:42.699007+0000","last_became_active":"2024-05-03T20:33:40.716010+0000","last_became_peered":"2024-05-03T20:33:40.716010+0000","last_unstale":"2024-05-03T20:33:42.699007+0000","last_undegraded":"2024-05-03T20:33:42.699007+0000","last_fullsized":"2024-05-03T20:33:42.699007+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:29:08.459822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.715739+0000","last_change":"2024-05-03T20:33:40.712105+0000","last_active":"2024-05-03T20:33:42.715739+0000","last_peered":"2024-05-03T20:33:42.715739+0000","last_clean":"2024-05-03T20:33:42.715739+0000","last_became_active":"2024-05-03T20:33:40.711753+0000","last_became_peered":"2024-05-03T20:33:40.711753+0000","last_unstale":"2024-05-03T20:33:42.715739+0000","last_undegraded":"2024-05-03T20:33:42.715739+0000","last_fullsized":"2024-05-03T20:33:42.715739+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:29:14.895475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711734+0000","last_change":"2024-05-03T20:33:40.713143+0000","last_active":"2024-05-03T20:33:42.711734+0000","last_peered":"2024-05-03T20:33:42.711734+0000","last_clean":"2024-05-03T20:33:42.711734+0000","last_became_active":"2024-05-03T20:33:40.712776+0000","last_became_peered":"2024-05-03T20:33:40.712776+0000","last_unstale":"2024-05-03T20:33:42.711734+0000","last_undegraded":"2024-05-03T20:33:42.711734+0000","last_fullsized":"2024-05-03T20:33:42.711734+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:43:23.345103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.713743+0000","last_change":"2024-05-03T20:33:40.706181+0000","last_active":"2024-05-03T20:33:42.713743+0000","last_peered":"2024-05-03T20:33:42.713743+0000","last_clean":"2024-05-03T20:33:42.713743+0000","last_became_active":"2024-05-03T20:33:40.705808+0000","last_became_peered":"2024-05-03T20:33:40.705808+0000","last_unstale":"2024-05-03T20:33:42.713743+0000","last_undegraded":"2024-05-03T20:33:42.713743+0000","last_fullsized":"2024-05-03T20:33:42.713743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:26:00.035916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.711519+0000","last_change":"2024-05-03T20:33:40.708950+0000","last_active":"2024-05-03T20:33:42.711519+0000","last_peered":"2024-05-03T20:33:42.711519+0000","last_clean":"2024-05-03T20:33:42.711519+0000","last_became_active":"2024-05-03T20:33:40.708578+0000","last_became_peered":"2024-05-03T20:33:40.708578+0000","last_unstale":"2024-05-03T20:33:42.711519+0000","last_undegraded":"2024-05-03T20:33:42.711519+0000","last_fullsized":"2024-05-03T20:33:42.711519+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:35:12.767718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.712210+0000","last_change":"2024-05-03T20:33:40.715949+0000","last_active":"2024-05-03T20:33:42.712210+0000","last_peered":"2024-05-03T20:33:42.712210+0000","last_clean":"2024-05-03T20:33:42.712210+0000","last_became_active":"2024-05-03T20:33:40.715366+0000","last_became_peered":"2024-05-03T20:33:40.715366+0000","last_unstale":"2024-05-03T20:33:42.712210+0000","last_undegraded":"2024-05-03T20:33:42.712210+0000","last_fullsized":"2024-05-03T20:33:42.712210+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:32:54.485990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.698650+0000","last_change":"2024-05-03T20:33:40.710447+0000","last_active":"2024-05-03T20:33:42.698650+0000","last_peered":"2024-05-03T20:33:42.698650+0000","last_clean":"2024-05-03T20:33:42.698650+0000","last_became_active":"2024-05-03T20:33:40.710107+0000","last_became_peered":"2024-05-03T20:33:40.710107+0000","last_unstale":"2024-05-03T20:33:42.698650+0000","last_undegraded":"2024-05-03T20:33:42.698650+0000","last_fullsized":"2024-05-03T20:33:42.698650+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:47:43.678947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-03T20:33:42.699004+0000","last_change":"2024-05-03T20:33:40.714081+0000","last_active":"2024-05-03T20:33:42.699004+0000","last_peered":"2024-05-03T20:33:42.699004+0000","last_clean":"2024-05-03T20:33:42.699004+0000","last_became_active":"2024-05-03T20:33:40.713678+0000","last_became_peered":"2024-05-03T20:33:40.713678+0000","last_unstale":"2024-05-03T20:33:42.699004+0000","last_undegraded":"2024-05-03T20:33:42.699004+0000","last_fullsized":"2024-05-03T20:33:42.699004+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:24:30.128065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":57196,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":280,"num_read_kb":206,"num_write":238,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":171588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612630,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":876544,"data_stored":421109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808860,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1062130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972386,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":421039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168614,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":421417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397546,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28476,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716676,"statfs":{"total":95995035648,"available":95965876224,"internally_reserved":0,"allocated":1495040,"data_stored":1014717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561072,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":421147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":471661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":905216,"data_stored":422095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":954368,"data_stored":471881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117054,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":420581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920825,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":421008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757301,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1507328,"data_stored":1013613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":51744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":51964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":51728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-03T20:34:26.282 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-03T20:34:26.283 INFO:tasks.cephadm:Setting up client nodes... 2024-05-03T20:34:26.283 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-03T20:34:26.526 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:34:26.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:26 smithi196 ceph-mon[32858]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T20:34:26.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:26 smithi204 ceph-mon[33595]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T20:34:26.928 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:26 smithi181 ceph-mon[29801]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T20:34:27.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:27 smithi196 ceph-mon[32858]: pgmap v307: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-03T20:34:27.860 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:27 smithi181 ceph-mon[29801]: pgmap v307: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-03T20:34:27.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:27 smithi204 ceph-mon[33595]: pgmap v307: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-03T20:34:28.064 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.062+0000 7f8905df0700 1 -- 172.21.15.181:0/2676633952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8900103fc0 msgr2=0x7f890010a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:28.064 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.062+0000 7f8905df0700 1 --2- 172.21.15.181:0/2676633952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8900103fc0 0x7f890010a870 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f88f800b0d0 tx=0x7f88f800b3e0 comp rx=0 tx=0).stop 2024-05-03T20:34:28.064 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.062+0000 7f8905df0700 1 -- 172.21.15.181:0/2676633952 shutdown_connections 2024-05-03T20:34:28.064 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.062+0000 7f8905df0700 1 --2- 172.21.15.181:0/2676633952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8900103fc0 0x7f890010a870 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:28.065 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.062+0000 7f8905df0700 1 --2- 172.21.15.181:0/2676633952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8900102420 0x7f8900102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:28.065 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.062+0000 7f8905df0700 1 --2- 172.21.15.181:0/2676633952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8900103620 0x7f8900103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:28.065 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.062+0000 7f8905df0700 1 -- 172.21.15.181:0/2676633952 >> 172.21.15.181:0/2676633952 conn(0x7f89000fd9c0 msgr2=0x7f89000ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:28.065 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.062+0000 7f8905df0700 1 -- 172.21.15.181:0/2676633952 shutdown_connections 2024-05-03T20:34:28.065 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.062+0000 7f8905df0700 1 -- 172.21.15.181:0/2676633952 wait complete. 2024-05-03T20:34:28.065 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.063+0000 7f8905df0700 1 Processor -- start 2024-05-03T20:34:28.065 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.063+0000 7f8905df0700 1 -- start start 2024-05-03T20:34:28.065 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.063+0000 7f8905df0700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8900102420 0x7f890006bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:28.065 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.063+0000 7f8905df0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8900103620 0x7f890019e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:28.065 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.063+0000 7f8905df0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8900103fc0 0x7f89001a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:28.065 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.064+0000 7f8905df0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f890010dec0 con 0x7f8900103620 2024-05-03T20:34:28.065 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.064+0000 7f8905df0700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f890010dd40 con 0x7f8900102420 2024-05-03T20:34:28.065 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.064+0000 7f8905df0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f890010e040 con 0x7f8900103fc0 2024-05-03T20:34:28.065 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.064+0000 7f88feffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8900103620 0x7f890019e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:28.065 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.064+0000 7f88feffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8900103620 0x7f890019e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45030/0 (socket says 172.21.15.181:45030) 2024-05-03T20:34:28.066 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.064+0000 7f88feffd700 1 -- 172.21.15.181:0/928376803 learned_addr learned my addr 172.21.15.181:0/928376803 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:34:28.066 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.064+0000 7f88fffff700 1 --2- 172.21.15.181:0/928376803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8900103fc0 0x7f89001a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:28.066 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.064+0000 7f88ff7fe700 1 --2- 172.21.15.181:0/928376803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8900102420 0x7f890006bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:28.066 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.064+0000 7f88feffd700 1 -- 172.21.15.181:0/928376803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8900103fc0 msgr2=0x7f89001a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:28.066 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.064+0000 7f88feffd700 1 --2- 172.21.15.181:0/928376803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8900103fc0 0x7f89001a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:28.066 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.064+0000 7f88feffd700 1 -- 172.21.15.181:0/928376803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8900102420 msgr2=0x7f890006bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:28.066 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.064+0000 7f88feffd700 1 --2- 172.21.15.181:0/928376803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8900102420 0x7f890006bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:28.066 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.064+0000 7f88feffd700 1 -- 172.21.15.181:0/928376803 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89001a1150 con 0x7f8900103620 2024-05-03T20:34:28.066 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.064+0000 7f88feffd700 1 --2- 172.21.15.181:0/928376803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8900103620 0x7f890019e1f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f88f000c970 tx=0x7f88f000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:28.066 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.064+0000 7f88fcff9700 1 -- 172.21.15.181:0/928376803 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f0007aa0 con 0x7f8900103620 2024-05-03T20:34:28.066 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.064+0000 7f88fcff9700 1 -- 172.21.15.181:0/928376803 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88f0004ce0 con 0x7f8900103620 2024-05-03T20:34:28.066 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.064+0000 7f8905df0700 1 -- 172.21.15.181:0/928376803 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89001a1440 con 0x7f8900103620 2024-05-03T20:34:28.066 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.064+0000 7f88fcff9700 1 -- 172.21.15.181:0/928376803 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f0005510 con 0x7f8900103620 2024-05-03T20:34:28.066 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.064+0000 7f8905df0700 1 -- 172.21.15.181:0/928376803 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89001a1930 con 0x7f8900103620 2024-05-03T20:34:28.067 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.065+0000 7f8905df0700 1 -- 172.21.15.181:0/928376803 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89000519e0 con 0x7f8900103620 2024-05-03T20:34:28.067 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.066+0000 7f88fcff9700 1 -- 172.21.15.181:0/928376803 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f88f0007c40 con 0x7f8900103620 2024-05-03T20:34:28.068 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.067+0000 7f88fcff9700 1 --2- 172.21.15.181:0/928376803 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f88e80b29b0 0x7f88e80b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:28.068 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.067+0000 7f88ff7fe700 1 --2- 172.21.15.181:0/928376803 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f88e80b29b0 0x7f88e80b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:28.068 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.068+0000 7f88fcff9700 1 -- 172.21.15.181:0/928376803 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f88f00e2310 con 0x7f8900103620 2024-05-03T20:34:28.068 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.068+0000 7f88ff7fe700 1 --2- 172.21.15.181:0/928376803 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f88e80b29b0 0x7f88e80b4e70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f88f4005fd0 tx=0x7f88f4005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:28.070 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.071+0000 7f88fcff9700 1 -- 172.21.15.181:0/928376803 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f88f00aa0a0 con 0x7f8900103620 2024-05-03T20:34:28.365 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.365+0000 7f8905df0700 1 -- 172.21.15.181:0/928376803 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f890002e5e0 con 0x7f8900103620 2024-05-03T20:34:28.367 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.368+0000 7f88fcff9700 1 -- 172.21.15.181:0/928376803 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f890002e5e0 con 0x7f8900103620 2024-05-03T20:34:28.367 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2024-05-03T20:34:28.367 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQBUSjVmqZPpFRAATBDVDzhBk73wvAPo2y5yFw== 2024-05-03T20:34:28.370 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.370+0000 7f8905df0700 1 -- 172.21.15.181:0/928376803 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f88e80b29b0 msgr2=0x7f88e80b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:28.370 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.370+0000 7f8905df0700 1 --2- 172.21.15.181:0/928376803 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f88e80b29b0 0x7f88e80b4e70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f88f4005fd0 tx=0x7f88f4005dd0 comp rx=0 tx=0).stop 2024-05-03T20:34:28.370 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.370+0000 7f8905df0700 1 -- 172.21.15.181:0/928376803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8900103620 msgr2=0x7f890019e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:28.370 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.370+0000 7f8905df0700 1 --2- 172.21.15.181:0/928376803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8900103620 0x7f890019e1f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f88f000c970 tx=0x7f88f000cc80 comp rx=0 tx=0).stop 2024-05-03T20:34:28.370 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.370+0000 7f8905df0700 1 -- 172.21.15.181:0/928376803 shutdown_connections 2024-05-03T20:34:28.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.370+0000 7f8905df0700 1 --2- 172.21.15.181:0/928376803 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f88e80b29b0 0x7f88e80b4e70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:28.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.370+0000 7f8905df0700 1 --2- 172.21.15.181:0/928376803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8900103fc0 0x7f89001a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:28.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.370+0000 7f8905df0700 1 --2- 172.21.15.181:0/928376803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8900102420 0x7f890006bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:28.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.370+0000 7f8905df0700 1 --2- 172.21.15.181:0/928376803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8900103620 0x7f890019e1f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:28.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.370+0000 7f8905df0700 1 -- 172.21.15.181:0/928376803 >> 172.21.15.181:0/928376803 conn(0x7f89000fd9c0 msgr2=0x7f89001068c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:28.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.371+0000 7f8905df0700 1 -- 172.21.15.181:0/928376803 shutdown_connections 2024-05-03T20:34:28.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:28.371+0000 7f8905df0700 1 -- 172.21.15.181:0/928376803 wait complete. 2024-05-03T20:34:28.668 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:28 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/928376803' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-03T20:34:28.668 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:28 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/928376803' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-03T20:34:28.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:28 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/928376803' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-03T20:34:28.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:28 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/928376803' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-03T20:34:28.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:28 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/928376803' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-03T20:34:28.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:28 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/928376803' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-03T20:34:29.069 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-03T20:34:29.069 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-03T20:34:29.069 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-03T20:34:29.105 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-03T20:34:29.105 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-03T20:34:29.105 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-03T20:34:29.384 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:34:29.810 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:29 smithi181 ceph-mon[29801]: pgmap v308: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-03T20:34:29.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:29 smithi196 ceph-mon[32858]: pgmap v308: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-03T20:34:29.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:29 smithi204 ceph-mon[33595]: pgmap v308: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-03T20:34:31.011 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.010+0000 7f6772b18700 1 -- 172.21.15.181:0/3188942690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f676c102fe0 msgr2=0x7f676c106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:31.011 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.010+0000 7f6772b18700 1 --2- 172.21.15.181:0/3188942690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f676c102fe0 0x7f676c106fe0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f67580099c0 tx=0x7f6758009cd0 comp rx=0 tx=0).stop 2024-05-03T20:34:31.012 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.010+0000 7f6772b18700 1 -- 172.21.15.181:0/3188942690 shutdown_connections 2024-05-03T20:34:31.012 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.010+0000 7f6772b18700 1 --2- 172.21.15.181:0/3188942690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f676c1076c0 0x7f676c10f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:31.012 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.010+0000 7f6772b18700 1 --2- 172.21.15.181:0/3188942690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f676c102630 0x7f676c102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:31.012 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.010+0000 7f6772b18700 1 --2- 172.21.15.181:0/3188942690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f676c102fe0 0x7f676c106fe0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:31.012 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.010+0000 7f6772b18700 1 -- 172.21.15.181:0/3188942690 >> 172.21.15.181:0/3188942690 conn(0x7f676c079350 msgr2=0x7f676c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:31.012 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.010+0000 7f6772b18700 1 -- 172.21.15.181:0/3188942690 shutdown_connections 2024-05-03T20:34:31.012 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.010+0000 7f6772b18700 1 -- 172.21.15.181:0/3188942690 wait complete. 2024-05-03T20:34:31.012 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.010+0000 7f6772b18700 1 Processor -- start 2024-05-03T20:34:31.012 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.010+0000 7f6772b18700 1 -- start start 2024-05-03T20:34:31.012 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.011+0000 7f6772b18700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f676c102630 0x7f676c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:31.012 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.011+0000 7f6772b18700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f676c102fe0 0x7f676c19af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:31.012 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.011+0000 7f6772b18700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f676c1076c0 0x7f676c1a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:31.012 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.011+0000 7f6772b18700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f676c112550 con 0x7f676c102630 2024-05-03T20:34:31.012 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.011+0000 7f6772b18700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f676c1123d0 con 0x7f676c102fe0 2024-05-03T20:34:31.012 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.011+0000 7f6772b18700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f676c1126d0 con 0x7f676c1076c0 2024-05-03T20:34:31.013 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.011+0000 7f67708b4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f676c102630 0x7f676c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:31.013 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.011+0000 7f67708b4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f676c102630 0x7f676c19aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45040/0 (socket says 172.21.15.181:45040) 2024-05-03T20:34:31.013 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.011+0000 7f67708b4700 1 -- 172.21.15.181:0/489057888 learned_addr learned my addr 172.21.15.181:0/489057888 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:34:31.013 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.011+0000 7f676bfff700 1 --2- 172.21.15.181:0/489057888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f676c102fe0 0x7f676c19af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:31.013 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.011+0000 7f67710b5700 1 --2- 172.21.15.181:0/489057888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f676c1076c0 0x7f676c1a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:31.013 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.011+0000 7f67708b4700 1 -- 172.21.15.181:0/489057888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f676c1076c0 msgr2=0x7f676c1a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:31.013 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.011+0000 7f67708b4700 1 --2- 172.21.15.181:0/489057888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f676c1076c0 0x7f676c1a5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:31.013 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.011+0000 7f67708b4700 1 -- 172.21.15.181:0/489057888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f676c102fe0 msgr2=0x7f676c19af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:31.013 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.011+0000 7f67708b4700 1 --2- 172.21.15.181:0/489057888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f676c102fe0 0x7f676c19af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:31.013 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.011+0000 7f67708b4700 1 -- 172.21.15.181:0/489057888 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676c1a5ff0 con 0x7f676c102630 2024-05-03T20:34:31.013 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.011+0000 7f676bfff700 1 --2- 172.21.15.181:0/489057888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f676c102fe0 0x7f676c19af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:34:31.013 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.011+0000 7f67708b4700 1 --2- 172.21.15.181:0/489057888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f676c102630 0x7f676c19aa30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f676000c920 tx=0x7f676000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:31.013 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.011+0000 7f6769ffb700 1 -- 172.21.15.181:0/489057888 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760007b00 con 0x7f676c102630 2024-05-03T20:34:31.013 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.011+0000 7f6772b18700 1 -- 172.21.15.181:0/489057888 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f676c1a61c0 con 0x7f676c102630 2024-05-03T20:34:31.014 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.012+0000 7f6769ffb700 1 -- 172.21.15.181:0/489057888 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6760004ce0 con 0x7f676c102630 2024-05-03T20:34:31.014 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.012+0000 7f6769ffb700 1 -- 172.21.15.181:0/489057888 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760005650 con 0x7f676c102630 2024-05-03T20:34:31.014 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.012+0000 7f6772b18700 1 -- 172.21.15.181:0/489057888 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f676c1a6720 con 0x7f676c102630 2024-05-03T20:34:31.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.013+0000 7f6772b18700 1 -- 172.21.15.181:0/489057888 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f676c06a190 con 0x7f676c102630 2024-05-03T20:34:31.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.014+0000 7f6769ffb700 1 -- 172.21.15.181:0/489057888 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f6760007ca0 con 0x7f676c102630 2024-05-03T20:34:31.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.014+0000 7f6769ffb700 1 --2- 172.21.15.181:0/489057888 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f675c0b2990 0x7f675c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:31.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.015+0000 7f676bfff700 1 --2- 172.21.15.181:0/489057888 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f675c0b2990 0x7f675c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:31.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.015+0000 7f6769ffb700 1 -- 172.21.15.181:0/489057888 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f67600e2190 con 0x7f676c102630 2024-05-03T20:34:31.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.015+0000 7f676bfff700 1 --2- 172.21.15.181:0/489057888 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f675c0b2990 0x7f675c0b4e50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6758000c00 tx=0x7f6758005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:31.017 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.018+0000 7f6769ffb700 1 -- 172.21.15.181:0/489057888 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6760014030 con 0x7f676c102630 2024-05-03T20:34:31.286 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.286+0000 7f6772b18700 1 -- 172.21.15.181:0/489057888 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f676c06c190 con 0x7f676c102630 2024-05-03T20:34:31.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.292+0000 7f6769ffb700 1 -- 172.21.15.181:0/489057888 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288656 (secure 0 0 0) 0x7f67600a9f80 con 0x7f676c102630 2024-05-03T20:34:31.295 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:34:31.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.295+0000 7f6772b18700 1 -- 172.21.15.181:0/489057888 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f675c0b2990 msgr2=0x7f675c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:31.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.295+0000 7f6772b18700 1 --2- 172.21.15.181:0/489057888 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f675c0b2990 0x7f675c0b4e50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6758000c00 tx=0x7f6758005fd0 comp rx=0 tx=0).stop 2024-05-03T20:34:31.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.295+0000 7f6772b18700 1 -- 172.21.15.181:0/489057888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f676c102630 msgr2=0x7f676c19aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:31.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.295+0000 7f6772b18700 1 --2- 172.21.15.181:0/489057888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f676c102630 0x7f676c19aa30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f676000c920 tx=0x7f676000cce0 comp rx=0 tx=0).stop 2024-05-03T20:34:31.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.295+0000 7f6772b18700 1 -- 172.21.15.181:0/489057888 shutdown_connections 2024-05-03T20:34:31.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.295+0000 7f6772b18700 1 --2- 172.21.15.181:0/489057888 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f675c0b2990 0x7f675c0b4e50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:31.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.295+0000 7f6772b18700 1 --2- 172.21.15.181:0/489057888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f676c1076c0 0x7f676c1a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:31.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.295+0000 7f6772b18700 1 --2- 172.21.15.181:0/489057888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f676c102630 0x7f676c19aa30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:31.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.295+0000 7f6772b18700 1 --2- 172.21.15.181:0/489057888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f676c102fe0 0x7f676c19af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:31.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.295+0000 7f6772b18700 1 -- 172.21.15.181:0/489057888 >> 172.21.15.181:0/489057888 conn(0x7f676c079350 msgr2=0x7f676c100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:31.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.295+0000 7f6772b18700 1 -- 172.21.15.181:0/489057888 shutdown_connections 2024-05-03T20:34:31.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:31.295+0000 7f6772b18700 1 -- 172.21.15.181:0/489057888 wait complete. 2024-05-03T20:34:31.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:31 smithi196 ceph-mon[32858]: pgmap v309: 97 pgs: 97 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-05-03T20:34:31.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:31 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/489057888' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-03T20:34:31.857 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:31 smithi181 ceph-mon[29801]: pgmap v309: 97 pgs: 97 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-05-03T20:34:31.857 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:31 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/489057888' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-03T20:34:31.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:31 smithi204 ceph-mon[33595]: pgmap v309: 97 pgs: 97 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-05-03T20:34:31.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:31 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/489057888' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-03T20:34:31.886 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1919423525},{"type":"v1","addr":"172.21.15.181:6801","nonce":1919423525}]},"active_addr":"172.21.15.181:6801/1919423525","active_change":"2024-05-03T20:26:33.766885+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.181:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":2051828443}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":4227551751}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":2203617391}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":2478476168}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":2533765669}]}]} 2024-05-03T20:34:31.893 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-03T20:34:31.893 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-03T20:34:31.893 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T20:34:32.135 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:34:33.703 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.698+0000 7fb65f29f700 1 -- 172.21.15.181:0/3286170723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb658102fe0 msgr2=0x7fb658106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:33.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.698+0000 7fb65f29f700 1 --2- 172.21.15.181:0/3286170723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb658102fe0 0x7fb658106fe0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb6480099c0 tx=0x7fb648009cd0 comp rx=0 tx=0).stop 2024-05-03T20:34:33.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.698+0000 7fb65f29f700 1 -- 172.21.15.181:0/3286170723 shutdown_connections 2024-05-03T20:34:33.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.698+0000 7fb65f29f700 1 --2- 172.21.15.181:0/3286170723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6581076c0 0x7fb65810f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:33.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.698+0000 7fb65f29f700 1 --2- 172.21.15.181:0/3286170723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb658102630 0x7fb658102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:33.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.698+0000 7fb65f29f700 1 --2- 172.21.15.181:0/3286170723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb658102fe0 0x7fb658106fe0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:33.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.698+0000 7fb65f29f700 1 -- 172.21.15.181:0/3286170723 >> 172.21.15.181:0/3286170723 conn(0x7fb658079370 msgr2=0x7fb658079780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:33.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.698+0000 7fb65f29f700 1 -- 172.21.15.181:0/3286170723 shutdown_connections 2024-05-03T20:34:33.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.698+0000 7fb65f29f700 1 -- 172.21.15.181:0/3286170723 wait complete. 2024-05-03T20:34:33.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.699+0000 7fb65f29f700 1 Processor -- start 2024-05-03T20:34:33.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.699+0000 7fb65f29f700 1 -- start start 2024-05-03T20:34:33.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.699+0000 7fb65f29f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb658102630 0x7fb65819f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:33.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.699+0000 7fb65f29f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb658102fe0 0x7fb65819fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:33.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.699+0000 7fb65f29f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6581076c0 0x7fb658199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:33.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.699+0000 7fb65f29f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb658112630 con 0x7fb658102fe0 2024-05-03T20:34:33.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.699+0000 7fb65f29f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581124b0 con 0x7fb658102630 2024-05-03T20:34:33.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.699+0000 7fb65f29f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581127b0 con 0x7fb6581076c0 2024-05-03T20:34:33.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.699+0000 7fb65c83a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb658102fe0 0x7fb65819fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:33.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.699+0000 7fb65c83a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb658102fe0 0x7fb65819fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45058/0 (socket says 172.21.15.181:45058) 2024-05-03T20:34:33.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.699+0000 7fb65c83a700 1 -- 172.21.15.181:0/3020797298 learned_addr learned my addr 172.21.15.181:0/3020797298 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:34:33.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.699+0000 7fb65d03b700 1 --2- 172.21.15.181:0/3020797298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb658102630 0x7fb65819f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:33.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.699+0000 7fb65d83c700 1 --2- 172.21.15.181:0/3020797298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6581076c0 0x7fb658199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:33.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.699+0000 7fb65c83a700 1 -- 172.21.15.181:0/3020797298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6581076c0 msgr2=0x7fb658199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:33.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.699+0000 7fb65c83a700 1 --2- 172.21.15.181:0/3020797298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6581076c0 0x7fb658199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:33.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.699+0000 7fb65c83a700 1 -- 172.21.15.181:0/3020797298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb658102630 msgr2=0x7fb65819f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:33.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.699+0000 7fb65c83a700 1 --2- 172.21.15.181:0/3020797298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb658102630 0x7fb65819f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:33.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.699+0000 7fb65c83a700 1 -- 172.21.15.181:0/3020797298 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65819a200 con 0x7fb658102fe0 2024-05-03T20:34:33.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.699+0000 7fb65d83c700 1 --2- 172.21.15.181:0/3020797298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6581076c0 0x7fb658199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:34:33.706 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.700+0000 7fb65c83a700 1 --2- 172.21.15.181:0/3020797298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb658102fe0 0x7fb65819fe20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb648009f50 tx=0x7fb6480037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:33.706 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.700+0000 7fb64e7fc700 1 -- 172.21.15.181:0/3020797298 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648027070 con 0x7fb658102fe0 2024-05-03T20:34:33.706 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.700+0000 7fb65f29f700 1 -- 172.21.15.181:0/3020797298 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb65819a490 con 0x7fb658102fe0 2024-05-03T20:34:33.706 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.700+0000 7fb64e7fc700 1 -- 172.21.15.181:0/3020797298 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb648004140 con 0x7fb658102fe0 2024-05-03T20:34:33.706 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.700+0000 7fb64e7fc700 1 -- 172.21.15.181:0/3020797298 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64800f460 con 0x7fb658102fe0 2024-05-03T20:34:33.706 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.700+0000 7fb65f29f700 1 -- 172.21.15.181:0/3020797298 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6581a67b0 con 0x7fb658102fe0 2024-05-03T20:34:33.706 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.701+0000 7fb65f29f700 1 -- 172.21.15.181:0/3020797298 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb65806a190 con 0x7fb658102fe0 2024-05-03T20:34:33.706 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.702+0000 7fb64e7fc700 1 -- 172.21.15.181:0/3020797298 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fb64802a070 con 0x7fb658102fe0 2024-05-03T20:34:33.706 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.703+0000 7fb64e7fc700 1 --2- 172.21.15.181:0/3020797298 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb6440b2910 0x7fb6440b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:33.706 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.703+0000 7fb65d03b700 1 --2- 172.21.15.181:0/3020797298 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb6440b2910 0x7fb6440b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:33.706 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.703+0000 7fb64e7fc700 1 -- 172.21.15.181:0/3020797298 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fb6480e3690 con 0x7fb658102fe0 2024-05-03T20:34:33.706 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.704+0000 7fb65d03b700 1 --2- 172.21.15.181:0/3020797298 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb6440b2910 0x7fb6440b4dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb654006fd0 tx=0x7fb654008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:33.707 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.707+0000 7fb64e7fc700 1 -- 172.21.15.181:0/3020797298 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb648013070 con 0x7fb658102fe0 2024-05-03T20:34:33.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:33 smithi196 ceph-mon[32858]: pgmap v310: 97 pgs: 97 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-03T20:34:33.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:33 smithi204 ceph-mon[33595]: pgmap v310: 97 pgs: 97 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-03T20:34:33.918 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.918+0000 7fb65f29f700 1 -- 172.21.15.181:0/3020797298 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb65802e510 con 0x7fb658102fe0 2024-05-03T20:34:33.918 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:33 smithi181 ceph-mon[29801]: pgmap v310: 97 pgs: 97 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-03T20:34:33.919 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.920+0000 7fb64e7fc700 1 -- 172.21.15.181:0/3020797298 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22240 (secure 0 0 0) 0x7fb6480ab390 con 0x7fb658102fe0 2024-05-03T20:34:33.920 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:34:33.920 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":81,"fsid":"1cd058ee-098b-11ef-bc97-c7b262605968","created":"2024-05-03T20:25:39.427815+0000","modified":"2024-05-03T20:33:42.488002+0000","last_up_change":"2024-05-03T20:32:38.854730+0000","last_in_change":"2024-05-03T20:32:18.787388+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T20:29:01.855187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T20:33:35.737273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T20:33:39.337556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5f594d97-10a4-4f77-aebc-c00dbe92a4af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6803","nonce":1683973278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6805","nonce":1683973278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6809","nonce":1683973278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6807","nonce":1683973278}]},"public_addr":"172.21.15.181:6803/1683973278","cluster_addr":"172.21.15.181:6805/1683973278","heartbeat_back_addr":"172.21.15.181:6809/1683973278","heartbeat_front_addr":"172.21.15.181:6807/1683973278","state":["exists","up"]},{"osd":1,"uuid":"9726d641-8930-4b45-9192-6ee2b6bfcdcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6801","nonce":2136873375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6803","nonce":2136873375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6807","nonce":2136873375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6805","nonce":2136873375}]},"public_addr":"172.21.15.196:6801/2136873375","cluster_addr":"172.21.15.196:6803/2136873375","heartbeat_back_addr":"172.21.15.196:6807/2136873375","heartbeat_front_addr":"172.21.15.196:6805/2136873375","state":["exists","up"]},{"osd":2,"uuid":"3d5573cc-37c1-4c3b-a61e-2363ef3ff964","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6801","nonce":3291673137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6803","nonce":3291673137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6807","nonce":3291673137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6805","nonce":3291673137}]},"public_addr":"172.21.15.204:6801/3291673137","cluster_addr":"172.21.15.204:6803/3291673137","heartbeat_back_addr":"172.21.15.204:6807/3291673137","heartbeat_front_addr":"172.21.15.204:6805/3291673137","state":["exists","up"]},{"osd":3,"uuid":"b9b205e3-a335-4771-9a66-55cfe6a8c31a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6811","nonce":29064971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6813","nonce":29064971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6817","nonce":29064971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6815","nonce":29064971}]},"public_addr":"172.21.15.181:6811/29064971","cluster_addr":"172.21.15.181:6813/29064971","heartbeat_back_addr":"172.21.15.181:6817/29064971","heartbeat_front_addr":"172.21.15.181:6815/29064971","state":["exists","up"]},{"osd":4,"uuid":"550f01ef-1947-4f9f-b759-34e3ea831839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6809","nonce":1145408995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6811","nonce":1145408995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6815","nonce":1145408995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6813","nonce":1145408995}]},"public_addr":"172.21.15.196:6809/1145408995","cluster_addr":"172.21.15.196:6811/1145408995","heartbeat_back_addr":"172.21.15.196:6815/1145408995","heartbeat_front_addr":"172.21.15.196:6813/1145408995","state":["exists","up"]},{"osd":5,"uuid":"6b74cb3a-f39a-4131-8793-1d5286856aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6809","nonce":855680559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6811","nonce":855680559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6815","nonce":855680559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6813","nonce":855680559}]},"public_addr":"172.21.15.204:6809/855680559","cluster_addr":"172.21.15.204:6811/855680559","heartbeat_back_addr":"172.21.15.204:6815/855680559","heartbeat_front_addr":"172.21.15.204:6813/855680559","state":["exists","up"]},{"osd":6,"uuid":"dbe19bdb-08eb-40aa-842b-0710963d4497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6819","nonce":1561969338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6821","nonce":1561969338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6825","nonce":1561969338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6823","nonce":1561969338}]},"public_addr":"172.21.15.181:6819/1561969338","cluster_addr":"172.21.15.181:6821/1561969338","heartbeat_back_addr":"172.21.15.181:6825/1561969338","heartbeat_front_addr":"172.21.15.181:6823/1561969338","state":["exists","up"]},{"osd":7,"uuid":"016c145c-ad67-42ac-9775-ca2840790abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6817","nonce":824890163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6819","nonce":824890163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6823","nonce":824890163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6821","nonce":824890163}]},"public_addr":"172.21.15.196:6817/824890163","cluster_addr":"172.21.15.196:6819/824890163","heartbeat_back_addr":"172.21.15.196:6823/824890163","heartbeat_front_addr":"172.21.15.196:6821/824890163","state":["exists","up"]},{"osd":8,"uuid":"10235638-c1f1-41d3-8e24-f8b2344bfeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6817","nonce":3566909552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6819","nonce":3566909552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6823","nonce":3566909552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6821","nonce":3566909552}]},"public_addr":"172.21.15.204:6817/3566909552","cluster_addr":"172.21.15.204:6819/3566909552","heartbeat_back_addr":"172.21.15.204:6823/3566909552","heartbeat_front_addr":"172.21.15.204:6821/3566909552","state":["exists","up"]},{"osd":9,"uuid":"96ce5aa6-e6a9-4d83-8562-589ac94c8cd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6827","nonce":241693472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6829","nonce":241693472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6833","nonce":241693472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6831","nonce":241693472}]},"public_addr":"172.21.15.181:6827/241693472","cluster_addr":"172.21.15.181:6829/241693472","heartbeat_back_addr":"172.21.15.181:6833/241693472","heartbeat_front_addr":"172.21.15.181:6831/241693472","state":["exists","up"]},{"osd":10,"uuid":"d6c9ffab-731c-46d5-88cd-5f21ba5acfa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6825","nonce":3980645995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6827","nonce":3980645995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6831","nonce":3980645995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6829","nonce":3980645995}]},"public_addr":"172.21.15.196:6825/3980645995","cluster_addr":"172.21.15.196:6827/3980645995","heartbeat_back_addr":"172.21.15.196:6831/3980645995","heartbeat_front_addr":"172.21.15.196:6829/3980645995","state":["exists","up"]},{"osd":11,"uuid":"3f58a2d6-bbbb-4e4d-bf83-2c886dd9484b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6825","nonce":488710574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6827","nonce":488710574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6831","nonce":488710574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6829","nonce":488710574}]},"public_addr":"172.21.15.204:6825/488710574","cluster_addr":"172.21.15.204:6827/488710574","heartbeat_back_addr":"172.21.15.204:6831/488710574","heartbeat_front_addr":"172.21.15.204:6829/488710574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:28:09.700091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:28:33.838568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:28:57.265536+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:29:21.720066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:29:45.480447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:30:09.558384+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:30:34.253331+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:30:57.986111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:31:22.470746+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:31:47.548762+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:32:11.285657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:32:36.888004+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6800/2389231557":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/3398276284":"2024-05-04T20:26:06.561713+0000","172.21.15.181:6801/2389231557":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/2327733609":"2024-05-04T20:26:06.561713+0000","172.21.15.181:0/628540759":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/3387163329":"2024-05-04T20:26:06.561713+0000","172.21.15.181:6800/1802984663":"2024-05-04T20:26:06.561713+0000","172.21.15.181:0/67924109":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/3693389657":"2024-05-04T20:26:33.766772+0000","172.21.15.181:6801/1802984663":"2024-05-04T20:26:06.561713+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T20:34:33.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.922+0000 7fb65f29f700 1 -- 172.21.15.181:0/3020797298 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb6440b2910 msgr2=0x7fb6440b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:33.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.922+0000 7fb65f29f700 1 --2- 172.21.15.181:0/3020797298 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb6440b2910 0x7fb6440b4dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb654006fd0 tx=0x7fb654008040 comp rx=0 tx=0).stop 2024-05-03T20:34:33.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.922+0000 7fb65f29f700 1 -- 172.21.15.181:0/3020797298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb658102fe0 msgr2=0x7fb65819fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:33.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.922+0000 7fb65f29f700 1 --2- 172.21.15.181:0/3020797298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb658102fe0 0x7fb65819fe20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb648009f50 tx=0x7fb6480037c0 comp rx=0 tx=0).stop 2024-05-03T20:34:33.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.923+0000 7fb65f29f700 1 -- 172.21.15.181:0/3020797298 shutdown_connections 2024-05-03T20:34:33.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.923+0000 7fb65f29f700 1 --2- 172.21.15.181:0/3020797298 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb6440b2910 0x7fb6440b4dd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:33.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.923+0000 7fb65f29f700 1 --2- 172.21.15.181:0/3020797298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6581076c0 0x7fb658199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:33.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.923+0000 7fb65f29f700 1 --2- 172.21.15.181:0/3020797298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb658102630 0x7fb65819f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:33.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.923+0000 7fb65f29f700 1 --2- 172.21.15.181:0/3020797298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb658102fe0 0x7fb65819fe20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:33.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.923+0000 7fb65f29f700 1 -- 172.21.15.181:0/3020797298 >> 172.21.15.181:0/3020797298 conn(0x7fb658079370 msgr2=0x7fb65806bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:33.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.923+0000 7fb65f29f700 1 -- 172.21.15.181:0/3020797298 shutdown_connections 2024-05-03T20:34:33.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:33.923+0000 7fb65f29f700 1 -- 172.21.15.181:0/3020797298 wait complete. 2024-05-03T20:34:34.480 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-03T20:34:34.480 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T20:34:34.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:34 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-05-03T20:34:35.538 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:34 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-05-03T20:34:35.538 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:34 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 6, 7, 11]}]: dispatch 2024-05-03T20:34:35.538 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:34 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-05-03T20:34:35.538 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:34 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]: dispatch 2024-05-03T20:34:35.538 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:34 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-05-03T20:34:35.538 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:34 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-05-03T20:34:35.538 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:34 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-03T20:34:35.538 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:34 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3020797298' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T20:34:35.543 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:34 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-05-03T20:34:35.543 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:34 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-05-03T20:34:35.543 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:34 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 6, 7, 11]}]: dispatch 2024-05-03T20:34:35.543 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:34 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-05-03T20:34:35.543 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:34 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]: dispatch 2024-05-03T20:34:35.543 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:34 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-05-03T20:34:35.543 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:34 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-05-03T20:34:35.543 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:34 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-03T20:34:35.543 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:34 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3020797298' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T20:34:35.544 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:34 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-05-03T20:34:35.544 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:34 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-05-03T20:34:35.544 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:34 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 6, 7, 11]}]: dispatch 2024-05-03T20:34:35.544 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:34 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-05-03T20:34:35.544 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:34 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]: dispatch 2024-05-03T20:34:35.544 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:34 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-05-03T20:34:35.544 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:34 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-05-03T20:34:35.544 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:34 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-03T20:34:35.544 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:34 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3020797298' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T20:34:35.544 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:34:35.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:35 smithi196 ceph-mon[32858]: pgmap v311: 97 pgs: 97 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-03T20:34:35.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:35 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-05-03T20:34:35.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:35 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-05-03T20:34:35.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:35 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 6, 7, 11]}]': finished 2024-05-03T20:34:35.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:35 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-05-03T20:34:35.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:35 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]': finished 2024-05-03T20:34:35.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:35 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-05-03T20:34:35.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:35 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-05-03T20:34:35.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:35 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-03T20:34:35.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:35 smithi196 ceph-mon[32858]: osdmap e82: 12 total, 12 up, 12 in 2024-05-03T20:34:35.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:35 smithi204 ceph-mon[33595]: pgmap v311: 97 pgs: 97 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-03T20:34:35.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:35 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-05-03T20:34:35.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:35 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-05-03T20:34:35.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:35 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 6, 7, 11]}]': finished 2024-05-03T20:34:35.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:35 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-05-03T20:34:35.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:35 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]': finished 2024-05-03T20:34:35.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:35 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-05-03T20:34:35.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:35 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-05-03T20:34:35.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:35 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-03T20:34:35.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:35 smithi204 ceph-mon[33595]: osdmap e82: 12 total, 12 up, 12 in 2024-05-03T20:34:35.941 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:35 smithi181 ceph-mon[29801]: pgmap v311: 97 pgs: 97 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-03T20:34:35.942 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:35 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-05-03T20:34:35.942 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:35 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-05-03T20:34:35.942 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:35 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 6, 7, 11]}]': finished 2024-05-03T20:34:35.942 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:35 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-05-03T20:34:35.942 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:35 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]': finished 2024-05-03T20:34:35.942 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:35 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-05-03T20:34:35.942 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:35 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-05-03T20:34:35.942 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:35 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-03T20:34:35.942 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:35 smithi181 ceph-mon[29801]: osdmap e82: 12 total, 12 up, 12 in 2024-05-03T20:34:36.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.281+0000 7f9de7dee700 1 -- 172.21.15.181:0/2811940116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9de0104160 msgr2=0x7f9de0108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:36.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.281+0000 7f9de7dee700 1 --2- 172.21.15.181:0/2811940116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9de0104160 0x7f9de0108150 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00099c0 tx=0x7f9dd0009cd0 comp rx=0 tx=0).stop 2024-05-03T20:34:36.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.281+0000 7f9de7dee700 1 -- 172.21.15.181:0/2811940116 shutdown_connections 2024-05-03T20:34:36.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.281+0000 7f9de7dee700 1 --2- 172.21.15.181:0/2811940116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9de0108880 0x7f9de0110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:36.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.281+0000 7f9de7dee700 1 --2- 172.21.15.181:0/2811940116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9de01037b0 0x7f9de0103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:36.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.281+0000 7f9de7dee700 1 --2- 172.21.15.181:0/2811940116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9de0104160 0x7f9de0108150 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:36.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.281+0000 7f9de7dee700 1 -- 172.21.15.181:0/2811940116 >> 172.21.15.181:0/2811940116 conn(0x7f9de007a580 msgr2=0x7f9de007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:36.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.281+0000 7f9de7dee700 1 -- 172.21.15.181:0/2811940116 shutdown_connections 2024-05-03T20:34:36.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.281+0000 7f9de7dee700 1 -- 172.21.15.181:0/2811940116 wait complete. 2024-05-03T20:34:36.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.282+0000 7f9de7dee700 1 Processor -- start 2024-05-03T20:34:36.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.282+0000 7f9de7dee700 1 -- start start 2024-05-03T20:34:36.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.282+0000 7f9de7dee700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9de01037b0 0x7f9de00757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:36.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.282+0000 7f9de7dee700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9de0104160 0x7f9de0075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:36.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.282+0000 7f9de7dee700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9de0108880 0x7f9de0076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:36.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.282+0000 7f9de7dee700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9de01136c0 con 0x7f9de01037b0 2024-05-03T20:34:36.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.282+0000 7f9de7dee700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9de0113540 con 0x7f9de0104160 2024-05-03T20:34:36.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.282+0000 7f9de7dee700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9de0113840 con 0x7f9de0108880 2024-05-03T20:34:36.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.282+0000 7f9de5b8a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9de01037b0 0x7f9de00757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:36.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.282+0000 7f9de5b8a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9de01037b0 0x7f9de00757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50076/0 (socket says 172.21.15.181:50076) 2024-05-03T20:34:36.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.282+0000 7f9de5b8a700 1 -- 172.21.15.181:0/2692158422 learned_addr learned my addr 172.21.15.181:0/2692158422 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:34:36.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.282+0000 7f9de5389700 1 --2- 172.21.15.181:0/2692158422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9de0104160 0x7f9de0075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:36.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.282+0000 7f9de638b700 1 --2- 172.21.15.181:0/2692158422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9de0108880 0x7f9de0076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:36.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.282+0000 7f9de5b8a700 1 -- 172.21.15.181:0/2692158422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9de0108880 msgr2=0x7f9de0076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:36.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.282+0000 7f9de5b8a700 1 --2- 172.21.15.181:0/2692158422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9de0108880 0x7f9de0076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:36.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.282+0000 7f9de5b8a700 1 -- 172.21.15.181:0/2692158422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9de0104160 msgr2=0x7f9de0075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:36.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.282+0000 7f9de5b8a700 1 --2- 172.21.15.181:0/2692158422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9de0104160 0x7f9de0075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:36.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.282+0000 7f9de5b8a700 1 -- 172.21.15.181:0/2692158422 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de01a8060 con 0x7f9de01037b0 2024-05-03T20:34:36.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.282+0000 7f9de5b8a700 1 --2- 172.21.15.181:0/2692158422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9de01037b0 0x7f9de00757a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80060b0 tx=0x7f9dd800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:36.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.283+0000 7f9dd6ffd700 1 -- 172.21.15.181:0/2692158422 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd801a070 con 0x7f9de01037b0 2024-05-03T20:34:36.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.283+0000 7f9dd6ffd700 1 -- 172.21.15.181:0/2692158422 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dd800ddd0 con 0x7f9de01037b0 2024-05-03T20:34:36.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.283+0000 7f9de7dee700 1 -- 172.21.15.181:0/2692158422 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de01a8350 con 0x7f9de01037b0 2024-05-03T20:34:36.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.283+0000 7f9dd6ffd700 1 -- 172.21.15.181:0/2692158422 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8014720 con 0x7f9de01037b0 2024-05-03T20:34:36.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.283+0000 7f9de7dee700 1 -- 172.21.15.181:0/2692158422 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de01a8860 con 0x7f9de01037b0 2024-05-03T20:34:36.286 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.284+0000 7f9de7dee700 1 -- 172.21.15.181:0/2692158422 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9de006a190 con 0x7f9de01037b0 2024-05-03T20:34:36.286 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.285+0000 7f9dd6ffd700 1 -- 172.21.15.181:0/2692158422 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f9dd801d070 con 0x7f9de01037b0 2024-05-03T20:34:36.286 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.285+0000 7f9dd6ffd700 1 --2- 172.21.15.181:0/2692158422 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f9dcc0b2910 0x7f9dcc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:36.286 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.286+0000 7f9de5389700 1 --2- 172.21.15.181:0/2692158422 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f9dcc0b2910 0x7f9dcc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:36.286 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.286+0000 7f9dd6ffd700 1 -- 172.21.15.181:0/2692158422 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f9dd80e20e0 con 0x7f9de01037b0 2024-05-03T20:34:36.286 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.286+0000 7f9de5389700 1 --2- 172.21.15.181:0/2692158422 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f9dcc0b2910 0x7f9dcc0b4dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009990 tx=0x7f9dd000b790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:36.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.289+0000 7f9dd6ffd700 1 -- 172.21.15.181:0/2692158422 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9dd8010070 con 0x7f9de01037b0 2024-05-03T20:34:36.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.498+0000 7f9de7dee700 1 -- 172.21.15.181:0/2692158422 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9de002e510 con 0x7f9de01037b0 2024-05-03T20:34:36.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.500+0000 7f9dd6ffd700 1 -- 172.21.15.181:0/2692158422 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22710 (secure 0 0 0) 0x7f9dd80a9ca0 con 0x7f9de01037b0 2024-05-03T20:34:36.499 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:34:36.500 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":83,"fsid":"1cd058ee-098b-11ef-bc97-c7b262605968","created":"2024-05-03T20:25:39.427815+0000","modified":"2024-05-03T20:34:35.581238+0000","last_up_change":"2024-05-03T20:32:38.854730+0000","last_in_change":"2024-05-03T20:32:18.787388+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T20:29:01.855187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T20:33:35.737273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T20:33:39.337556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5f594d97-10a4-4f77-aebc-c00dbe92a4af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6803","nonce":1683973278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6805","nonce":1683973278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6809","nonce":1683973278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6807","nonce":1683973278}]},"public_addr":"172.21.15.181:6803/1683973278","cluster_addr":"172.21.15.181:6805/1683973278","heartbeat_back_addr":"172.21.15.181:6809/1683973278","heartbeat_front_addr":"172.21.15.181:6807/1683973278","state":["exists","up"]},{"osd":1,"uuid":"9726d641-8930-4b45-9192-6ee2b6bfcdcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6801","nonce":2136873375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6803","nonce":2136873375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6807","nonce":2136873375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6805","nonce":2136873375}]},"public_addr":"172.21.15.196:6801/2136873375","cluster_addr":"172.21.15.196:6803/2136873375","heartbeat_back_addr":"172.21.15.196:6807/2136873375","heartbeat_front_addr":"172.21.15.196:6805/2136873375","state":["exists","up"]},{"osd":2,"uuid":"3d5573cc-37c1-4c3b-a61e-2363ef3ff964","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6801","nonce":3291673137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6803","nonce":3291673137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6807","nonce":3291673137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6805","nonce":3291673137}]},"public_addr":"172.21.15.204:6801/3291673137","cluster_addr":"172.21.15.204:6803/3291673137","heartbeat_back_addr":"172.21.15.204:6807/3291673137","heartbeat_front_addr":"172.21.15.204:6805/3291673137","state":["exists","up"]},{"osd":3,"uuid":"b9b205e3-a335-4771-9a66-55cfe6a8c31a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6811","nonce":29064971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6813","nonce":29064971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6817","nonce":29064971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6815","nonce":29064971}]},"public_addr":"172.21.15.181:6811/29064971","cluster_addr":"172.21.15.181:6813/29064971","heartbeat_back_addr":"172.21.15.181:6817/29064971","heartbeat_front_addr":"172.21.15.181:6815/29064971","state":["exists","up"]},{"osd":4,"uuid":"550f01ef-1947-4f9f-b759-34e3ea831839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6809","nonce":1145408995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6811","nonce":1145408995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6815","nonce":1145408995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6813","nonce":1145408995}]},"public_addr":"172.21.15.196:6809/1145408995","cluster_addr":"172.21.15.196:6811/1145408995","heartbeat_back_addr":"172.21.15.196:6815/1145408995","heartbeat_front_addr":"172.21.15.196:6813/1145408995","state":["exists","up"]},{"osd":5,"uuid":"6b74cb3a-f39a-4131-8793-1d5286856aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6809","nonce":855680559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6811","nonce":855680559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6815","nonce":855680559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6813","nonce":855680559}]},"public_addr":"172.21.15.204:6809/855680559","cluster_addr":"172.21.15.204:6811/855680559","heartbeat_back_addr":"172.21.15.204:6815/855680559","heartbeat_front_addr":"172.21.15.204:6813/855680559","state":["exists","up"]},{"osd":6,"uuid":"dbe19bdb-08eb-40aa-842b-0710963d4497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6819","nonce":1561969338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6821","nonce":1561969338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6825","nonce":1561969338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6823","nonce":1561969338}]},"public_addr":"172.21.15.181:6819/1561969338","cluster_addr":"172.21.15.181:6821/1561969338","heartbeat_back_addr":"172.21.15.181:6825/1561969338","heartbeat_front_addr":"172.21.15.181:6823/1561969338","state":["exists","up"]},{"osd":7,"uuid":"016c145c-ad67-42ac-9775-ca2840790abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6817","nonce":824890163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6819","nonce":824890163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6823","nonce":824890163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6821","nonce":824890163}]},"public_addr":"172.21.15.196:6817/824890163","cluster_addr":"172.21.15.196:6819/824890163","heartbeat_back_addr":"172.21.15.196:6823/824890163","heartbeat_front_addr":"172.21.15.196:6821/824890163","state":["exists","up"]},{"osd":8,"uuid":"10235638-c1f1-41d3-8e24-f8b2344bfeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6817","nonce":3566909552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6819","nonce":3566909552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6823","nonce":3566909552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6821","nonce":3566909552}]},"public_addr":"172.21.15.204:6817/3566909552","cluster_addr":"172.21.15.204:6819/3566909552","heartbeat_back_addr":"172.21.15.204:6823/3566909552","heartbeat_front_addr":"172.21.15.204:6821/3566909552","state":["exists","up"]},{"osd":9,"uuid":"96ce5aa6-e6a9-4d83-8562-589ac94c8cd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6827","nonce":241693472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6829","nonce":241693472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6833","nonce":241693472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6831","nonce":241693472}]},"public_addr":"172.21.15.181:6827/241693472","cluster_addr":"172.21.15.181:6829/241693472","heartbeat_back_addr":"172.21.15.181:6833/241693472","heartbeat_front_addr":"172.21.15.181:6831/241693472","state":["exists","up"]},{"osd":10,"uuid":"d6c9ffab-731c-46d5-88cd-5f21ba5acfa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6825","nonce":3980645995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6827","nonce":3980645995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6831","nonce":3980645995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6829","nonce":3980645995}]},"public_addr":"172.21.15.196:6825/3980645995","cluster_addr":"172.21.15.196:6827/3980645995","heartbeat_back_addr":"172.21.15.196:6831/3980645995","heartbeat_front_addr":"172.21.15.196:6829/3980645995","state":["exists","up"]},{"osd":11,"uuid":"3f58a2d6-bbbb-4e4d-bf83-2c886dd9484b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6825","nonce":488710574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6827","nonce":488710574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6831","nonce":488710574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6829","nonce":488710574}]},"public_addr":"172.21.15.204:6825/488710574","cluster_addr":"172.21.15.204:6827/488710574","heartbeat_back_addr":"172.21.15.204:6831/488710574","heartbeat_front_addr":"172.21.15.204:6829/488710574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:28:09.700091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:28:33.838568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:28:57.265536+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:29:21.720066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:29:45.480447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:30:09.558384+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:30:34.253331+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:30:57.986111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:31:22.470746+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:31:47.548762+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:32:11.285657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:32:36.888004+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":6},{"from":7,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6800/2389231557":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/3398276284":"2024-05-04T20:26:06.561713+0000","172.21.15.181:6801/2389231557":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/2327733609":"2024-05-04T20:26:06.561713+0000","172.21.15.181:0/628540759":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/3387163329":"2024-05-04T20:26:06.561713+0000","172.21.15.181:6800/1802984663":"2024-05-04T20:26:06.561713+0000","172.21.15.181:0/67924109":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/3693389657":"2024-05-04T20:26:33.766772+0000","172.21.15.181:6801/1802984663":"2024-05-04T20:26:06.561713+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T20:34:36.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.502+0000 7f9de7dee700 1 -- 172.21.15.181:0/2692158422 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f9dcc0b2910 msgr2=0x7f9dcc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:36.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.502+0000 7f9de7dee700 1 --2- 172.21.15.181:0/2692158422 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f9dcc0b2910 0x7f9dcc0b4dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009990 tx=0x7f9dd000b790 comp rx=0 tx=0).stop 2024-05-03T20:34:36.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.502+0000 7f9de7dee700 1 -- 172.21.15.181:0/2692158422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9de01037b0 msgr2=0x7f9de00757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:36.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.502+0000 7f9de7dee700 1 --2- 172.21.15.181:0/2692158422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9de01037b0 0x7f9de00757a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80060b0 tx=0x7f9dd800d3b0 comp rx=0 tx=0).stop 2024-05-03T20:34:36.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.502+0000 7f9de7dee700 1 -- 172.21.15.181:0/2692158422 shutdown_connections 2024-05-03T20:34:36.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.502+0000 7f9de7dee700 1 --2- 172.21.15.181:0/2692158422 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f9dcc0b2910 0x7f9dcc0b4dd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:36.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.502+0000 7f9de7dee700 1 --2- 172.21.15.181:0/2692158422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9de0108880 0x7f9de0076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:36.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.502+0000 7f9de7dee700 1 --2- 172.21.15.181:0/2692158422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9de01037b0 0x7f9de00757a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:36.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.502+0000 7f9de7dee700 1 --2- 172.21.15.181:0/2692158422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9de0104160 0x7f9de0075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:36.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.502+0000 7f9de7dee700 1 -- 172.21.15.181:0/2692158422 >> 172.21.15.181:0/2692158422 conn(0x7f9de007a580 msgr2=0x7f9de0102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:36.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.502+0000 7f9de7dee700 1 -- 172.21.15.181:0/2692158422 shutdown_connections 2024-05-03T20:34:36.503 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:36.502+0000 7f9de7dee700 1 -- 172.21.15.181:0/2692158422 wait complete. 2024-05-03T20:34:36.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:36 smithi196 ceph-mon[32858]: osdmap e83: 12 total, 12 up, 12 in 2024-05-03T20:34:36.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:36 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/2692158422' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T20:34:36.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:36 smithi204 ceph-mon[33595]: osdmap e83: 12 total, 12 up, 12 in 2024-05-03T20:34:36.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:36 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/2692158422' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T20:34:36.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:36 smithi181 ceph-mon[29801]: osdmap e83: 12 total, 12 up, 12 in 2024-05-03T20:34:36.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:36 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/2692158422' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T20:34:37.147 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-03T20:34:37.147 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-03T20:34:37.148 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-03T20:34:37.148 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-03T20:34:37.148 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-03T20:34:37.149 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-03T20:34:37.149 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-03T20:34:37.150 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-03T20:34:37.150 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-03T20:34:37.151 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-03T20:34:37.151 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-03T20:34:37.152 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-03T20:34:37.796 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:34:37.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:37 smithi196 ceph-mon[32858]: pgmap v314: 97 pgs: 1 peering, 96 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 639 B/s wr, 10 op/s 2024-05-03T20:34:37.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:37 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:34:37.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:37 smithi204 ceph-mon[33595]: pgmap v314: 97 pgs: 1 peering, 96 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 639 B/s wr, 10 op/s 2024-05-03T20:34:37.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:37 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:34:37.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:37 smithi181 ceph-mon[29801]: pgmap v314: 97 pgs: 1 peering, 96 active+clean; 636 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 639 B/s wr, 10 op/s 2024-05-03T20:34:37.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:37 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:34:38.086 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:34:38.097 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:34:38.097 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:34:38.404 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:34:38.553 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:34:38.553 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:34:38.555 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:34:39.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:39 smithi196 ceph-mon[32858]: pgmap v315: 97 pgs: 1 peering, 96 active+clean; 636 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4 op/s; 146 B/s, 0 objects/s recovering 2024-05-03T20:34:39.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:39 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:34:39.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:39 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:34:39.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:39 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:34:39.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:39 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:34:39.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:39 smithi204 ceph-mon[33595]: pgmap v315: 97 pgs: 1 peering, 96 active+clean; 636 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4 op/s; 146 B/s, 0 objects/s recovering 2024-05-03T20:34:39.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:39 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:34:39.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:39 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:34:39.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:39 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:34:39.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:39 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:34:39.638 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:39 smithi181 ceph-mon[29801]: pgmap v315: 97 pgs: 1 peering, 96 active+clean; 636 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4 op/s; 146 B/s, 0 objects/s recovering 2024-05-03T20:34:39.638 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:39 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:34:39.638 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:39 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:34:39.638 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:39 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:34:39.638 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:39 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:34:41.008 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:34:41.009 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:34:41.460 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:41 smithi181 ceph-mon[29801]: pgmap v316: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 5 op/s; 146 B/s, 1 keys/s, 1 objects/s recovering 2024-05-03T20:34:41.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:41 smithi196 ceph-mon[32858]: pgmap v316: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 5 op/s; 146 B/s, 1 keys/s, 1 objects/s recovering 2024-05-03T20:34:41.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:41 smithi204 ceph-mon[33595]: pgmap v316: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 5 op/s; 146 B/s, 1 keys/s, 1 objects/s recovering 2024-05-03T20:34:43.554 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:43 smithi181 ceph-mon[29801]: pgmap v317: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 3 op/s; 146 B/s, 1 keys/s, 1 objects/s recovering 2024-05-03T20:34:43.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:43 smithi196 ceph-mon[32858]: pgmap v317: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 3 op/s; 146 B/s, 1 keys/s, 1 objects/s recovering 2024-05-03T20:34:43.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:43 smithi204 ceph-mon[33595]: pgmap v317: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 3 op/s; 146 B/s, 1 keys/s, 1 objects/s recovering 2024-05-03T20:34:45.525 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.398+0000 7fb61d695700 1 -- 172.21.15.181:0/3214091638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb618102d60 msgr2=0x7fb61810f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:45.525 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.398+0000 7fb61d695700 1 --2- 172.21.15.181:0/3214091638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb618102d60 0x7fb61810f250 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb61000b0a0 tx=0x7fb61000b3b0 comp rx=0 tx=0).stop 2024-05-03T20:34:45.525 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.398+0000 7fb61d695700 1 -- 172.21.15.181:0/3214091638 shutdown_connections 2024-05-03T20:34:45.525 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.398+0000 7fb61d695700 1 --2- 172.21.15.181:0/3214091638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb618102d60 0x7fb61810f250 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:45.525 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.398+0000 7fb61d695700 1 --2- 172.21.15.181:0/3214091638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb618108410 0x7fb6181087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:45.525 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.398+0000 7fb61d695700 1 --2- 172.21.15.181:0/3214091638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6181023a0 0x7fb618102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:45.526 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.398+0000 7fb61d695700 1 -- 172.21.15.181:0/3214091638 >> 172.21.15.181:0/3214091638 conn(0x7fb618079350 msgr2=0x7fb618079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:45.526 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.398+0000 7fb61d695700 1 -- 172.21.15.181:0/3214091638 shutdown_connections 2024-05-03T20:34:45.526 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.398+0000 7fb61d695700 1 -- 172.21.15.181:0/3214091638 wait complete. 2024-05-03T20:34:45.526 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.399+0000 7fb61d695700 1 Processor -- start 2024-05-03T20:34:45.526 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.399+0000 7fb61d695700 1 -- start start 2024-05-03T20:34:45.526 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.399+0000 7fb61d695700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6181023a0 0x7fb618104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:45.526 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.399+0000 7fb61d695700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb618102d60 0x7fb6181045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:45.526 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.399+0000 7fb61d695700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb618108410 0x7fb618104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:45.526 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.399+0000 7fb61d695700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb6181125b0 con 0x7fb618108410 2024-05-03T20:34:45.526 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.399+0000 7fb61d695700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb618112430 con 0x7fb618102d60 2024-05-03T20:34:45.526 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.399+0000 7fb61d695700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb618112730 con 0x7fb6181023a0 2024-05-03T20:34:45.526 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.400+0000 7fb6177fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb618108410 0x7fb618104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:45.526 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.400+0000 7fb6177fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb618108410 0x7fb618104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50100/0 (socket says 172.21.15.181:50100) 2024-05-03T20:34:45.526 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.400+0000 7fb6177fe700 1 -- 172.21.15.181:0/299142070 learned_addr learned my addr 172.21.15.181:0/299142070 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:34:45.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.400+0000 7fb616ffd700 1 --2- 172.21.15.181:0/299142070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6181023a0 0x7fb618104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:45.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.400+0000 7fb6167fc700 1 --2- 172.21.15.181:0/299142070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb618102d60 0x7fb6181045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:45.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.400+0000 7fb6177fe700 1 -- 172.21.15.181:0/299142070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6181023a0 msgr2=0x7fb618104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:45.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.400+0000 7fb6177fe700 1 --2- 172.21.15.181:0/299142070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6181023a0 0x7fb618104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:45.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.400+0000 7fb6177fe700 1 -- 172.21.15.181:0/299142070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb618102d60 msgr2=0x7fb6181045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:45.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.400+0000 7fb6177fe700 1 --2- 172.21.15.181:0/299142070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb618102d60 0x7fb6181045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:45.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.400+0000 7fb6177fe700 1 -- 172.21.15.181:0/299142070 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6181a8090 con 0x7fb618108410 2024-05-03T20:34:45.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.400+0000 7fb6177fe700 1 --2- 172.21.15.181:0/299142070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb618108410 0x7fb618104b00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb6100062a0 tx=0x7fb610004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:45.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.400+0000 7fb5fffff700 1 -- 172.21.15.181:0/299142070 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb610021070 con 0x7fb618108410 2024-05-03T20:34:45.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.400+0000 7fb61d695700 1 -- 172.21.15.181:0/299142070 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6181a8320 con 0x7fb618108410 2024-05-03T20:34:45.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.400+0000 7fb5fffff700 1 -- 172.21.15.181:0/299142070 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb610007c80 con 0x7fb618108410 2024-05-03T20:34:45.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.400+0000 7fb5fffff700 1 -- 172.21.15.181:0/299142070 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61001d450 con 0x7fb618108410 2024-05-03T20:34:45.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.400+0000 7fb61d695700 1 -- 172.21.15.181:0/299142070 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6181a8780 con 0x7fb618108410 2024-05-03T20:34:45.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.401+0000 7fb61d695700 1 -- 172.21.15.181:0/299142070 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb618106490 con 0x7fb618108410 2024-05-03T20:34:45.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.404+0000 7fb5fffff700 1 -- 172.21.15.181:0/299142070 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fb610004000 con 0x7fb618108410 2024-05-03T20:34:45.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.404+0000 7fb5fffff700 1 --2- 172.21.15.181:0/299142070 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb6000b2910 0x7fb6000b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:45.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.404+0000 7fb616ffd700 1 --2- 172.21.15.181:0/299142070 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb6000b2910 0x7fb6000b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:45.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.405+0000 7fb5fffff700 1 -- 172.21.15.181:0/299142070 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7fb6100e3990 con 0x7fb618108410 2024-05-03T20:34:45.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.405+0000 7fb5fffff700 1 --2- 172.21.15.181:0/299142070 >> [v2:172.21.15.196:6808/1145408995,v1:172.21.15.196:6809/1145408995] conn(0x7fb6000bc450 0x7fb6000be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:45.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.405+0000 7fb5fffff700 1 -- 172.21.15.181:0/299142070 --> [v2:172.21.15.196:6808/1145408995,v1:172.21.15.196:6809/1145408995] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb610009ce0 con 0x7fb6000bc450 2024-05-03T20:34:45.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.405+0000 7fb616ffd700 1 --2- 172.21.15.181:0/299142070 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb6000b2910 0x7fb6000b4dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb60c009a10 tx=0x7fb60c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:45.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.405+0000 7fb5fffff700 1 -- 172.21.15.181:0/299142070 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fb6100e3de0 con 0x7fb618108410 2024-05-03T20:34:45.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.405+0000 7fb6167fc700 1 --2- 172.21.15.181:0/299142070 >> [v2:172.21.15.196:6808/1145408995,v1:172.21.15.196:6809/1145408995] conn(0x7fb6000bc450 0x7fb6000be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:45.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.406+0000 7fb6167fc700 1 --2- 172.21.15.181:0/299142070 >> [v2:172.21.15.196:6808/1145408995,v1:172.21.15.196:6809/1145408995] conn(0x7fb6000bc450 0x7fb6000be8f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:45.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.407+0000 7fb5fffff700 1 -- 172.21.15.181:0/299142070 <== osd.4 v2:172.21.15.196:6808/1145408995 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb610009ce0 con 0x7fb6000bc450 2024-05-03T20:34:45.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.423+0000 7fb61d695700 1 -- 172.21.15.181:0/299142070 --> [v2:172.21.15.196:6808/1145408995,v1:172.21.15.196:6809/1145408995] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb618103ac0 con 0x7fb6000bc450 2024-05-03T20:34:45.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.424+0000 7fb5fffff700 1 -- 172.21.15.181:0/299142070 <== osd.4 v2:172.21.15.196:6808/1145408995 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb618103ac0 con 0x7fb6000bc450 2024-05-03T20:34:45.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.424+0000 7fb61d695700 1 -- 172.21.15.181:0/299142070 >> [v2:172.21.15.196:6808/1145408995,v1:172.21.15.196:6809/1145408995] conn(0x7fb6000bc450 msgr2=0x7fb6000be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:45.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.424+0000 7fb61d695700 1 --2- 172.21.15.181:0/299142070 >> [v2:172.21.15.196:6808/1145408995,v1:172.21.15.196:6809/1145408995] conn(0x7fb6000bc450 0x7fb6000be8f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:45.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.424+0000 7fb61d695700 1 -- 172.21.15.181:0/299142070 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb6000b2910 msgr2=0x7fb6000b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:45.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.424+0000 7fb61d695700 1 --2- 172.21.15.181:0/299142070 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb6000b2910 0x7fb6000b4dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb60c009a10 tx=0x7fb60c009190 comp rx=0 tx=0).stop 2024-05-03T20:34:45.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.424+0000 7fb61d695700 1 -- 172.21.15.181:0/299142070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb618108410 msgr2=0x7fb618104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:45.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.424+0000 7fb61d695700 1 --2- 172.21.15.181:0/299142070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb618108410 0x7fb618104b00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb6100062a0 tx=0x7fb610004e20 comp rx=0 tx=0).stop 2024-05-03T20:34:45.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.424+0000 7fb6177fe700 1 -- 172.21.15.181:0/299142070 reap_dead start 2024-05-03T20:34:45.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.424+0000 7fb61d695700 1 -- 172.21.15.181:0/299142070 shutdown_connections 2024-05-03T20:34:45.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.424+0000 7fb61d695700 1 -- 172.21.15.181:0/299142070 >> 172.21.15.181:0/299142070 conn(0x7fb618079350 msgr2=0x7fb618101640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:45.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.424+0000 7fb61d695700 1 -- 172.21.15.181:0/299142070 shutdown_connections 2024-05-03T20:34:45.530 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:44.424+0000 7fb61d695700 1 -- 172.21.15.181:0/299142070 wait complete. 2024-05-03T20:34:45.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:45 smithi204 ceph-mon[33595]: pgmap v318: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 6 op/s; 126 B/s, 1 keys/s, 1 objects/s recovering 2024-05-03T20:34:45.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:45 smithi181 ceph-mon[29801]: pgmap v318: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 6 op/s; 126 B/s, 1 keys/s, 1 objects/s recovering 2024-05-03T20:34:45.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:45 smithi196 ceph-mon[32858]: pgmap v318: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 6 op/s; 126 B/s, 1 keys/s, 1 objects/s recovering 2024-05-03T20:34:47.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:47 smithi181 ceph-mon[29801]: pgmap v319: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 6 op/s; 113 B/s, 0 keys/s, 1 objects/s recovering 2024-05-03T20:34:47.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:47 smithi204 ceph-mon[33595]: pgmap v319: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 6 op/s; 113 B/s, 0 keys/s, 1 objects/s recovering 2024-05-03T20:34:47.625 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.601+0000 7f59ce4f9700 1 -- 172.21.15.181:0/884704806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59c8103530 msgr2=0x7f59c81039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:47.625 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.601+0000 7f59ce4f9700 1 --2- 172.21.15.181:0/884704806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59c8103530 0x7f59c81039b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0099c0 tx=0x7f59bc009cd0 comp rx=0 tx=0).stop 2024-05-03T20:34:47.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.601+0000 7f59ce4f9700 1 -- 172.21.15.181:0/884704806 shutdown_connections 2024-05-03T20:34:47.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.601+0000 7f59ce4f9700 1 --2- 172.21.15.181:0/884704806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59c8103ef0 0x7f59c81103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:47.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.601+0000 7f59ce4f9700 1 --2- 172.21.15.181:0/884704806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59c81095d0 0x7f59c81099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:47.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.601+0000 7f59ce4f9700 1 --2- 172.21.15.181:0/884704806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59c8103530 0x7f59c81039b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:47.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.601+0000 7f59ce4f9700 1 -- 172.21.15.181:0/884704806 >> 172.21.15.181:0/884704806 conn(0x7f59c807a690 msgr2=0x7f59c807aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:47.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.601+0000 7f59ce4f9700 1 -- 172.21.15.181:0/884704806 shutdown_connections 2024-05-03T20:34:47.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.601+0000 7f59ce4f9700 1 -- 172.21.15.181:0/884704806 wait complete. 2024-05-03T20:34:47.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.602+0000 7f59ce4f9700 1 Processor -- start 2024-05-03T20:34:47.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.602+0000 7f59ce4f9700 1 -- start start 2024-05-03T20:34:47.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.602+0000 7f59ce4f9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59c8103530 0x7f59c819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:47.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.602+0000 7f59ce4f9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59c8103ef0 0x7f59c819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:47.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.602+0000 7f59ce4f9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59c81095d0 0x7f59c8199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:47.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.602+0000 7f59ce4f9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f59c81136d0 con 0x7f59c81095d0 2024-05-03T20:34:47.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.602+0000 7f59ce4f9700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f59c8113550 con 0x7f59c8103530 2024-05-03T20:34:47.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.602+0000 7f59ce4f9700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f59c8113850 con 0x7f59c8103ef0 2024-05-03T20:34:47.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.602+0000 7f59cca96700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59c81095d0 0x7f59c8199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:47.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.602+0000 7f59cca96700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59c81095d0 0x7f59c8199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59938/0 (socket says 172.21.15.181:59938) 2024-05-03T20:34:47.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.602+0000 7f59cca96700 1 -- 172.21.15.181:0/492888401 learned_addr learned my addr 172.21.15.181:0/492888401 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:34:47.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.602+0000 7f59c77fe700 1 --2- 172.21.15.181:0/492888401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59c8103ef0 0x7f59c819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:47.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.602+0000 7f59c7fff700 1 --2- 172.21.15.181:0/492888401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59c8103530 0x7f59c819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:47.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.602+0000 7f59cca96700 1 -- 172.21.15.181:0/492888401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59c8103ef0 msgr2=0x7f59c819fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:47.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.602+0000 7f59cca96700 1 --2- 172.21.15.181:0/492888401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59c8103ef0 0x7f59c819fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:47.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.603+0000 7f59cca96700 1 -- 172.21.15.181:0/492888401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59c8103530 msgr2=0x7f59c819f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:47.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.603+0000 7f59cca96700 1 --2- 172.21.15.181:0/492888401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59c8103530 0x7f59c819f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:47.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.603+0000 7f59cca96700 1 -- 172.21.15.181:0/492888401 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c819a190 con 0x7f59c81095d0 2024-05-03T20:34:47.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.603+0000 7f59cca96700 1 --2- 172.21.15.181:0/492888401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59c81095d0 0x7f59c8199a30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f59c000ada0 tx=0x7f59c000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:47.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.603+0000 7f59c57fa700 1 -- 172.21.15.181:0/492888401 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0017020 con 0x7f59c81095d0 2024-05-03T20:34:47.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.603+0000 7f59ce4f9700 1 -- 172.21.15.181:0/492888401 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c819a480 con 0x7f59c81095d0 2024-05-03T20:34:47.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.603+0000 7f59c57fa700 1 -- 172.21.15.181:0/492888401 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59c000cd80 con 0x7f59c81095d0 2024-05-03T20:34:47.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.603+0000 7f59c57fa700 1 -- 172.21.15.181:0/492888401 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0005280 con 0x7f59c81095d0 2024-05-03T20:34:47.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.603+0000 7f59ce4f9700 1 -- 172.21.15.181:0/492888401 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c81a66f0 con 0x7f59c81095d0 2024-05-03T20:34:47.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.604+0000 7f59ce4f9700 1 -- 172.21.15.181:0/492888401 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f59c8063bb0 con 0x7f59c81095d0 2024-05-03T20:34:47.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.605+0000 7f59c57fa700 1 -- 172.21.15.181:0/492888401 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f59c000a080 con 0x7f59c81095d0 2024-05-03T20:34:47.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.606+0000 7f59c57fa700 1 --2- 172.21.15.181:0/492888401 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f59b00b2910 0x7f59b00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:47.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.606+0000 7f59c7fff700 1 --2- 172.21.15.181:0/492888401 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f59b00b2910 0x7f59b00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:47.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.606+0000 7f59c57fa700 1 -- 172.21.15.181:0/492888401 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f59c00e2430 con 0x7f59c81095d0 2024-05-03T20:34:47.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.606+0000 7f59c57fa700 1 --2- 172.21.15.181:0/492888401 >> [v2:172.21.15.204:6816/3566909552,v1:172.21.15.204:6817/3566909552] conn(0x7f59b00bc450 0x7f59b00be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:47.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.606+0000 7f59c57fa700 1 -- 172.21.15.181:0/492888401 --> [v2:172.21.15.204:6816/3566909552,v1:172.21.15.204:6817/3566909552] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f59b00befc0 con 0x7f59b00bc450 2024-05-03T20:34:47.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.606+0000 7f59c57fa700 1 -- 172.21.15.181:0/492888401 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f59c00e2880 con 0x7f59c81095d0 2024-05-03T20:34:47.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.606+0000 7f59c7fff700 1 --2- 172.21.15.181:0/492888401 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f59b00b2910 0x7f59b00b4dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f59b8005950 tx=0x7f59b800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:47.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.606+0000 7f59c77fe700 1 --2- 172.21.15.181:0/492888401 >> [v2:172.21.15.204:6816/3566909552,v1:172.21.15.204:6817/3566909552] conn(0x7f59b00bc450 0x7f59b00be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:47.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.607+0000 7f59c77fe700 1 --2- 172.21.15.181:0/492888401 >> [v2:172.21.15.204:6816/3566909552,v1:172.21.15.204:6817/3566909552] conn(0x7f59b00bc450 0x7f59b00be8f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:47.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.608+0000 7f59c57fa700 1 -- 172.21.15.181:0/492888401 <== osd.8 v2:172.21.15.204:6816/3566909552 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f59b00befc0 con 0x7f59b00bc450 2024-05-03T20:34:47.629 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.624+0000 7f59ce4f9700 1 -- 172.21.15.181:0/492888401 --> [v2:172.21.15.204:6816/3566909552,v1:172.21.15.204:6817/3566909552] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f59c8064030 con 0x7f59b00bc450 2024-05-03T20:34:47.629 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.624+0000 7f59c57fa700 1 -- 172.21.15.181:0/492888401 <== osd.8 v2:172.21.15.204:6816/3566909552 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f59c8064030 con 0x7f59b00bc450 2024-05-03T20:34:47.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.624+0000 7f59ce4f9700 1 -- 172.21.15.181:0/492888401 >> [v2:172.21.15.204:6816/3566909552,v1:172.21.15.204:6817/3566909552] conn(0x7f59b00bc450 msgr2=0x7f59b00be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:47.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.624+0000 7f59ce4f9700 1 --2- 172.21.15.181:0/492888401 >> [v2:172.21.15.204:6816/3566909552,v1:172.21.15.204:6817/3566909552] conn(0x7f59b00bc450 0x7f59b00be8f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:47.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.625+0000 7f59ce4f9700 1 -- 172.21.15.181:0/492888401 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f59b00b2910 msgr2=0x7f59b00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:47.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.625+0000 7f59ce4f9700 1 --2- 172.21.15.181:0/492888401 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f59b00b2910 0x7f59b00b4dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f59b8005950 tx=0x7f59b800a300 comp rx=0 tx=0).stop 2024-05-03T20:34:47.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.625+0000 7f59ce4f9700 1 -- 172.21.15.181:0/492888401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59c81095d0 msgr2=0x7f59c8199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:47.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.625+0000 7f59ce4f9700 1 --2- 172.21.15.181:0/492888401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59c81095d0 0x7f59c8199a30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f59c000ada0 tx=0x7f59c000c330 comp rx=0 tx=0).stop 2024-05-03T20:34:47.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.625+0000 7f59cca96700 1 -- 172.21.15.181:0/492888401 reap_dead start 2024-05-03T20:34:47.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.625+0000 7f59ce4f9700 1 -- 172.21.15.181:0/492888401 shutdown_connections 2024-05-03T20:34:47.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.625+0000 7f59ce4f9700 1 -- 172.21.15.181:0/492888401 >> 172.21.15.181:0/492888401 conn(0x7f59c807a690 msgr2=0x7f59c8102240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:47.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.625+0000 7f59ce4f9700 1 -- 172.21.15.181:0/492888401 shutdown_connections 2024-05-03T20:34:47.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:47.625+0000 7f59ce4f9700 1 -- 172.21.15.181:0/492888401 wait complete. 2024-05-03T20:34:47.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:47 smithi196 ceph-mon[32858]: pgmap v319: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 6 op/s; 113 B/s, 0 keys/s, 1 objects/s recovering 2024-05-03T20:34:48.054 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:34:48.054 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:34:48.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.292+0000 7f03fc616700 1 -- 172.21.15.181:0/3885736194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03f4108400 msgr2=0x7f03f41087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:48.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.292+0000 7f03fc616700 1 --2- 172.21.15.181:0/3885736194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03f4108400 0x7f03f41087e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009990 tx=0x7f03f0009ca0 comp rx=0 tx=0).stop 2024-05-03T20:34:48.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.293+0000 7f03fc616700 1 -- 172.21.15.181:0/3885736194 shutdown_connections 2024-05-03T20:34:48.295 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.293+0000 7f03fc616700 1 --2- 172.21.15.181:0/3885736194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03f4102d50 0x7f03f410f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:48.295 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.293+0000 7f03fc616700 1 --2- 172.21.15.181:0/3885736194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03f4108400 0x7f03f41087e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:48.295 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.293+0000 7f03fc616700 1 --2- 172.21.15.181:0/3885736194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03f4102390 0x7f03f4102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:48.295 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.293+0000 7f03fc616700 1 -- 172.21.15.181:0/3885736194 >> 172.21.15.181:0/3885736194 conn(0x7f03f4079310 msgr2=0x7f03f4079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:48.295 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.293+0000 7f03fc616700 1 -- 172.21.15.181:0/3885736194 shutdown_connections 2024-05-03T20:34:48.295 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.293+0000 7f03fc616700 1 -- 172.21.15.181:0/3885736194 wait complete. 2024-05-03T20:34:48.295 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.293+0000 7f03fc616700 1 Processor -- start 2024-05-03T20:34:48.295 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.293+0000 7f03fc616700 1 -- start start 2024-05-03T20:34:48.295 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.293+0000 7f03fc616700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03f4102390 0x7f03f419f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:48.295 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.293+0000 7f03fc616700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03f4102d50 0x7f03f419fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:48.295 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.293+0000 7f03fc616700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03f4108400 0x7f03f41999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:48.295 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.293+0000 7f03fc616700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f03f4112520 con 0x7f03f4108400 2024-05-03T20:34:48.295 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.293+0000 7f03fc616700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41123a0 con 0x7f03f4102d50 2024-05-03T20:34:48.295 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.293+0000 7f03fc616700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41126a0 con 0x7f03f4102390 2024-05-03T20:34:48.295 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.294+0000 7f03fabb3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03f4108400 0x7f03f41999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:48.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.294+0000 7f03fabb3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03f4108400 0x7f03f41999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59952/0 (socket says 172.21.15.181:59952) 2024-05-03T20:34:48.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.294+0000 7f03fabb3700 1 -- 172.21.15.181:0/1136914834 learned_addr learned my addr 172.21.15.181:0/1136914834 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:34:48.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.294+0000 7f03fa3b2700 1 --2- 172.21.15.181:0/1136914834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03f4102390 0x7f03f419f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:48.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.294+0000 7f03f9bb1700 1 --2- 172.21.15.181:0/1136914834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03f4102d50 0x7f03f419fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:48.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.294+0000 7f03fabb3700 1 -- 172.21.15.181:0/1136914834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03f4102390 msgr2=0x7f03f419f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:48.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.294+0000 7f03fabb3700 1 --2- 172.21.15.181:0/1136914834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03f4102390 0x7f03f419f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:48.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.294+0000 7f03fabb3700 1 -- 172.21.15.181:0/1136914834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03f4102d50 msgr2=0x7f03f419fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:48.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.294+0000 7f03fabb3700 1 --2- 172.21.15.181:0/1136914834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03f4102d50 0x7f03f419fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:48.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.294+0000 7f03fabb3700 1 -- 172.21.15.181:0/1136914834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f419a150 con 0x7f03f4108400 2024-05-03T20:34:48.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.294+0000 7f03f9bb1700 1 --2- 172.21.15.181:0/1136914834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03f4102d50 0x7f03f419fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:34:48.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.294+0000 7f03fa3b2700 1 --2- 172.21.15.181:0/1136914834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03f4102390 0x7f03f419f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:34:48.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.294+0000 7f03fabb3700 1 --2- 172.21.15.181:0/1136914834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03f4108400 0x7f03f41999f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f03ec007ee0 tx=0x7f03ec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:48.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.294+0000 7f03eb7fe700 1 -- 172.21.15.181:0/1136914834 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec00dbd0 con 0x7f03f4108400 2024-05-03T20:34:48.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.294+0000 7f03eb7fe700 1 -- 172.21.15.181:0/1136914834 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03ec00dd70 con 0x7f03f4108400 2024-05-03T20:34:48.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.294+0000 7f03fc616700 1 -- 172.21.15.181:0/1136914834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f419a440 con 0x7f03f4108400 2024-05-03T20:34:48.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.294+0000 7f03eb7fe700 1 -- 172.21.15.181:0/1136914834 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec005990 con 0x7f03f4108400 2024-05-03T20:34:48.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.294+0000 7f03fc616700 1 -- 172.21.15.181:0/1136914834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f41a6700 con 0x7f03f4108400 2024-05-03T20:34:48.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.295+0000 7f03e97fa700 1 -- 172.21.15.181:0/1136914834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f03f4063bb0 con 0x7f03f4108400 2024-05-03T20:34:48.298 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.296+0000 7f03eb7fe700 1 -- 172.21.15.181:0/1136914834 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f03ec01a020 con 0x7f03f4108400 2024-05-03T20:34:48.298 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.297+0000 7f03eb7fe700 1 --2- 172.21.15.181:0/1136914834 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f03e00b2910 0x7f03e00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:48.298 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.297+0000 7f03fa3b2700 1 --2- 172.21.15.181:0/1136914834 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f03e00b2910 0x7f03e00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:48.298 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.297+0000 7f03eb7fe700 1 -- 172.21.15.181:0/1136914834 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f03ec0e21f0 con 0x7f03f4108400 2024-05-03T20:34:48.298 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.297+0000 7f03eb7fe700 1 --2- 172.21.15.181:0/1136914834 >> [v2:172.21.15.204:6800/3291673137,v1:172.21.15.204:6801/3291673137] conn(0x7f03e00bc450 0x7f03e00be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:48.298 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.297+0000 7f03eb7fe700 1 -- 172.21.15.181:0/1136914834 --> [v2:172.21.15.204:6800/3291673137,v1:172.21.15.204:6801/3291673137] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f03ec019e70 con 0x7f03e00bc450 2024-05-03T20:34:48.298 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.297+0000 7f03eb7fe700 1 -- 172.21.15.181:0/1136914834 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f03ec0e2640 con 0x7f03f4108400 2024-05-03T20:34:48.298 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.298+0000 7f03fa3b2700 1 --2- 172.21.15.181:0/1136914834 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f03e00b2910 0x7f03e00b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f03f0003800 tx=0x7f03f001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:48.298 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.298+0000 7f03f9bb1700 1 --2- 172.21.15.181:0/1136914834 >> [v2:172.21.15.204:6800/3291673137,v1:172.21.15.204:6801/3291673137] conn(0x7f03e00bc450 0x7f03e00be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:48.298 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.298+0000 7f03f9bb1700 1 --2- 172.21.15.181:0/1136914834 >> [v2:172.21.15.204:6800/3291673137,v1:172.21.15.204:6801/3291673137] conn(0x7f03e00bc450 0x7f03e00be8f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:48.299 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.299+0000 7f03eb7fe700 1 -- 172.21.15.181:0/1136914834 <== osd.2 v2:172.21.15.204:6800/3291673137 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03ec019e70 con 0x7f03e00bc450 2024-05-03T20:34:48.314 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.315+0000 7f03e97fa700 1 -- 172.21.15.181:0/1136914834 --> [v2:172.21.15.204:6800/3291673137,v1:172.21.15.204:6801/3291673137] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f03f4064030 con 0x7f03e00bc450 2024-05-03T20:34:48.315 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.316+0000 7f03eb7fe700 1 -- 172.21.15.181:0/1136914834 <== osd.2 v2:172.21.15.204:6800/3291673137 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f03f4064030 con 0x7f03e00bc450 2024-05-03T20:34:48.316 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.316+0000 7f03e97fa700 1 -- 172.21.15.181:0/1136914834 >> [v2:172.21.15.204:6800/3291673137,v1:172.21.15.204:6801/3291673137] conn(0x7f03e00bc450 msgr2=0x7f03e00be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:48.316 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.316+0000 7f03e97fa700 1 --2- 172.21.15.181:0/1136914834 >> [v2:172.21.15.204:6800/3291673137,v1:172.21.15.204:6801/3291673137] conn(0x7f03e00bc450 0x7f03e00be8f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:48.316 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.317+0000 7f03e97fa700 1 -- 172.21.15.181:0/1136914834 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f03e00b2910 msgr2=0x7f03e00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:48.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.317+0000 7f03e97fa700 1 --2- 172.21.15.181:0/1136914834 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f03e00b2910 0x7f03e00b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f03f0003800 tx=0x7f03f001a040 comp rx=0 tx=0).stop 2024-05-03T20:34:48.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.317+0000 7f03e97fa700 1 -- 172.21.15.181:0/1136914834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03f4108400 msgr2=0x7f03f41999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:48.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.317+0000 7f03e97fa700 1 --2- 172.21.15.181:0/1136914834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03f4108400 0x7f03f41999f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f03ec007ee0 tx=0x7f03ec00d3b0 comp rx=0 tx=0).stop 2024-05-03T20:34:48.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.317+0000 7f03fabb3700 1 -- 172.21.15.181:0/1136914834 reap_dead start 2024-05-03T20:34:48.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.317+0000 7f03e97fa700 1 -- 172.21.15.181:0/1136914834 shutdown_connections 2024-05-03T20:34:48.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.317+0000 7f03e97fa700 1 -- 172.21.15.181:0/1136914834 >> 172.21.15.181:0/1136914834 conn(0x7f03f4079310 msgr2=0x7f03f406bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:48.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.317+0000 7f03e97fa700 1 -- 172.21.15.181:0/1136914834 shutdown_connections 2024-05-03T20:34:48.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:48.317+0000 7f03e97fa700 1 -- 172.21.15.181:0/1136914834 wait complete. 2024-05-03T20:34:49.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:49 smithi181 ceph-mon[29801]: pgmap v320: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6 op/s; 97 B/s, 0 keys/s, 1 objects/s recovering 2024-05-03T20:34:49.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:49 smithi196 ceph-mon[32858]: pgmap v320: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6 op/s; 97 B/s, 0 keys/s, 1 objects/s recovering 2024-05-03T20:34:49.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:49 smithi204 ceph-mon[33595]: pgmap v320: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6 op/s; 97 B/s, 0 keys/s, 1 objects/s recovering 2024-05-03T20:34:49.992 INFO:teuthology.orchestra.run.smithi181.stdout:77309411402 2024-05-03T20:34:49.993 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-03T20:34:51.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.099+0000 7fa4637fb700 1 -- 172.21.15.181:0/965441906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa45c108870 msgr2=0x7fa45c1103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:51.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.099+0000 7fa4637fb700 1 --2- 172.21.15.181:0/965441906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa45c108870 0x7fa45c1103f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa45400b0d0 tx=0x7fa45400b3e0 comp rx=0 tx=0).stop 2024-05-03T20:34:51.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.099+0000 7fa4637fb700 1 -- 172.21.15.181:0/965441906 shutdown_connections 2024-05-03T20:34:51.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.099+0000 7fa4637fb700 1 --2- 172.21.15.181:0/965441906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa45c108870 0x7fa45c1103f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:51.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.099+0000 7fa4637fb700 1 --2- 172.21.15.181:0/965441906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa45c103780 0x7fa45c103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:51.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.099+0000 7fa4637fb700 1 --2- 172.21.15.181:0/965441906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa45c104130 0x7fa45c108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:51.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.099+0000 7fa4637fb700 1 -- 172.21.15.181:0/965441906 >> 172.21.15.181:0/965441906 conn(0x7fa45c07a610 msgr2=0x7fa45c07aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:51.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.099+0000 7fa4637fb700 1 -- 172.21.15.181:0/965441906 shutdown_connections 2024-05-03T20:34:51.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.100+0000 7fa4637fb700 1 -- 172.21.15.181:0/965441906 wait complete. 2024-05-03T20:34:51.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.100+0000 7fa4637fb700 1 Processor -- start 2024-05-03T20:34:51.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.100+0000 7fa4637fb700 1 -- start start 2024-05-03T20:34:51.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.100+0000 7fa4637fb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa45c103780 0x7fa45c19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:51.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.100+0000 7fa4637fb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa45c104130 0x7fa45c19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:51.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.100+0000 7fa4637fb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa45c108870 0x7fa45c1998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:51.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.100+0000 7fa4637fb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c1136b0 con 0x7fa45c108870 2024-05-03T20:34:51.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.100+0000 7fa4637fb700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c113530 con 0x7fa45c103780 2024-05-03T20:34:51.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.100+0000 7fa4637fb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c113830 con 0x7fa45c104130 2024-05-03T20:34:51.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.100+0000 7fa461d98700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa45c108870 0x7fa45c1998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:51.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.100+0000 7fa461d98700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa45c108870 0x7fa45c1998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60008/0 (socket says 172.21.15.181:60008) 2024-05-03T20:34:51.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.100+0000 7fa461d98700 1 -- 172.21.15.181:0/4005577713 learned_addr learned my addr 172.21.15.181:0/4005577713 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:34:51.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.101+0000 7fa460d96700 1 --2- 172.21.15.181:0/4005577713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa45c104130 0x7fa45c19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:51.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.101+0000 7fa461597700 1 --2- 172.21.15.181:0/4005577713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa45c103780 0x7fa45c19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:51.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.101+0000 7fa461d98700 1 -- 172.21.15.181:0/4005577713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa45c104130 msgr2=0x7fa45c19fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:51.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.101+0000 7fa461d98700 1 --2- 172.21.15.181:0/4005577713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa45c104130 0x7fa45c19fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:51.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.101+0000 7fa461d98700 1 -- 172.21.15.181:0/4005577713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa45c103780 msgr2=0x7fa45c19f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:51.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.101+0000 7fa461d98700 1 --2- 172.21.15.181:0/4005577713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa45c103780 0x7fa45c19f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:51.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.101+0000 7fa461d98700 1 -- 172.21.15.181:0/4005577713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa45c19a050 con 0x7fa45c108870 2024-05-03T20:34:51.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.101+0000 7fa460d96700 1 --2- 172.21.15.181:0/4005577713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa45c104130 0x7fa45c19fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:34:51.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.101+0000 7fa461597700 1 --2- 172.21.15.181:0/4005577713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa45c103780 0x7fa45c19f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:34:51.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.101+0000 7fa461d98700 1 --2- 172.21.15.181:0/4005577713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa45c108870 0x7fa45c1998f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa4540095c0 tx=0x7fa454007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:51.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.101+0000 7fa4527fc700 1 -- 172.21.15.181:0/4005577713 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45401f070 con 0x7fa45c108870 2024-05-03T20:34:51.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.101+0000 7fa4637fb700 1 -- 172.21.15.181:0/4005577713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa45c19a2e0 con 0x7fa45c108870 2024-05-03T20:34:51.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.101+0000 7fa4527fc700 1 -- 172.21.15.181:0/4005577713 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa454003c00 con 0x7fa45c108870 2024-05-03T20:34:51.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.101+0000 7fa4527fc700 1 -- 172.21.15.181:0/4005577713 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45401b6a0 con 0x7fa45c108870 2024-05-03T20:34:51.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.101+0000 7fa4637fb700 1 -- 172.21.15.181:0/4005577713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa45c1a6600 con 0x7fa45c108870 2024-05-03T20:34:51.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.102+0000 7fa4637fb700 1 -- 172.21.15.181:0/4005577713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa45c063c30 con 0x7fa45c108870 2024-05-03T20:34:51.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.103+0000 7fa4527fc700 1 -- 172.21.15.181:0/4005577713 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fa454022070 con 0x7fa45c108870 2024-05-03T20:34:51.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.104+0000 7fa4527fc700 1 --2- 172.21.15.181:0/4005577713 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fa4480b2910 0x7fa4480b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:51.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.104+0000 7fa461597700 1 --2- 172.21.15.181:0/4005577713 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fa4480b2910 0x7fa4480b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:51.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.104+0000 7fa4527fc700 1 -- 172.21.15.181:0/4005577713 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7fa45400f020 con 0x7fa45c108870 2024-05-03T20:34:51.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.104+0000 7fa4527fc700 1 --2- 172.21.15.181:0/4005577713 >> [v2:172.21.15.181:6810/29064971,v1:172.21.15.181:6811/29064971] conn(0x7fa4480bc450 0x7fa4480be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:51.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.104+0000 7fa4527fc700 1 -- 172.21.15.181:0/4005577713 --> [v2:172.21.15.181:6810/29064971,v1:172.21.15.181:6811/29064971] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa4480befc0 con 0x7fa4480bc450 2024-05-03T20:34:51.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.104+0000 7fa4527fc700 1 -- 172.21.15.181:0/4005577713 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fa4540e3d70 con 0x7fa45c108870 2024-05-03T20:34:51.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.104+0000 7fa460d96700 1 --2- 172.21.15.181:0/4005577713 >> [v2:172.21.15.181:6810/29064971,v1:172.21.15.181:6811/29064971] conn(0x7fa4480bc450 0x7fa4480be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:51.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.104+0000 7fa461597700 1 --2- 172.21.15.181:0/4005577713 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fa4480b2910 0x7fa4480b4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa45c19aa30 tx=0x7fa458009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:51.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.105+0000 7fa460d96700 1 --2- 172.21.15.181:0/4005577713 >> [v2:172.21.15.181:6810/29064971,v1:172.21.15.181:6811/29064971] conn(0x7fa4480bc450 0x7fa4480be8f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:51.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.105+0000 7fa4527fc700 1 -- 172.21.15.181:0/4005577713 <== osd.3 v2:172.21.15.181:6810/29064971 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa4480befc0 con 0x7fa4480bc450 2024-05-03T20:34:51.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.122+0000 7fa4637fb700 1 -- 172.21.15.181:0/4005577713 --> [v2:172.21.15.181:6810/29064971,v1:172.21.15.181:6811/29064971] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa45c064030 con 0x7fa4480bc450 2024-05-03T20:34:51.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.122+0000 7fa4527fc700 1 -- 172.21.15.181:0/4005577713 <== osd.3 v2:172.21.15.181:6810/29064971 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa45c064030 con 0x7fa4480bc450 2024-05-03T20:34:51.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.123+0000 7fa4637fb700 1 -- 172.21.15.181:0/4005577713 >> [v2:172.21.15.181:6810/29064971,v1:172.21.15.181:6811/29064971] conn(0x7fa4480bc450 msgr2=0x7fa4480be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:51.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.123+0000 7fa4637fb700 1 --2- 172.21.15.181:0/4005577713 >> [v2:172.21.15.181:6810/29064971,v1:172.21.15.181:6811/29064971] conn(0x7fa4480bc450 0x7fa4480be8f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:51.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.123+0000 7fa4637fb700 1 -- 172.21.15.181:0/4005577713 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fa4480b2910 msgr2=0x7fa4480b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:51.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.123+0000 7fa4637fb700 1 --2- 172.21.15.181:0/4005577713 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fa4480b2910 0x7fa4480b4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa45c19aa30 tx=0x7fa458009670 comp rx=0 tx=0).stop 2024-05-03T20:34:51.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.123+0000 7fa4637fb700 1 -- 172.21.15.181:0/4005577713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa45c108870 msgr2=0x7fa45c1998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:51.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.123+0000 7fa4637fb700 1 --2- 172.21.15.181:0/4005577713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa45c108870 0x7fa45c1998f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa4540095c0 tx=0x7fa454007ad0 comp rx=0 tx=0).stop 2024-05-03T20:34:51.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.123+0000 7fa461d98700 1 -- 172.21.15.181:0/4005577713 reap_dead start 2024-05-03T20:34:51.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.123+0000 7fa4637fb700 1 -- 172.21.15.181:0/4005577713 shutdown_connections 2024-05-03T20:34:51.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.123+0000 7fa4637fb700 1 -- 172.21.15.181:0/4005577713 >> 172.21.15.181:0/4005577713 conn(0x7fa45c07a610 msgr2=0x7fa45c1099c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:51.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.123+0000 7fa4637fb700 1 -- 172.21.15.181:0/4005577713 shutdown_connections 2024-05-03T20:34:51.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.123+0000 7fa4637fb700 1 -- 172.21.15.181:0/4005577713 wait complete. 2024-05-03T20:34:51.211 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.988+0000 7f51911db700 1 -- 172.21.15.181:0/2582429130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f518c103040 msgr2=0x7f518c107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:51.212 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.988+0000 7f51911db700 1 --2- 172.21.15.181:0/2582429130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f518c103040 0x7f518c107000 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f51800099c0 tx=0x7f5180009cd0 comp rx=0 tx=0).stop 2024-05-03T20:34:51.212 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.988+0000 7f51911db700 1 -- 172.21.15.181:0/2582429130 shutdown_connections 2024-05-03T20:34:51.212 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.988+0000 7f51911db700 1 --2- 172.21.15.181:0/2582429130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f518c1076e0 0x7f518c10f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:51.212 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.988+0000 7f51911db700 1 --2- 172.21.15.181:0/2582429130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f518c102690 0x7f518c102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:51.212 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.988+0000 7f51911db700 1 --2- 172.21.15.181:0/2582429130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f518c103040 0x7f518c107000 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:51.212 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.988+0000 7f51911db700 1 -- 172.21.15.181:0/2582429130 >> 172.21.15.181:0/2582429130 conn(0x7f518c0783f0 msgr2=0x7f518c078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:51.212 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.988+0000 7f51911db700 1 -- 172.21.15.181:0/2582429130 shutdown_connections 2024-05-03T20:34:51.212 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.988+0000 7f51911db700 1 -- 172.21.15.181:0/2582429130 wait complete. 2024-05-03T20:34:51.212 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.988+0000 7f51911db700 1 Processor -- start 2024-05-03T20:34:51.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.989+0000 7f51911db700 1 -- start start 2024-05-03T20:34:51.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.989+0000 7f51911db700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f518c102690 0x7f518c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:51.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.989+0000 7f51911db700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f518c103040 0x7f518c19af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:51.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.989+0000 7f51911db700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f518c1076e0 0x7f518c1a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:51.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.989+0000 7f51911db700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1125d0 con 0x7f518c1076e0 2024-05-03T20:34:51.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.989+0000 7f51911db700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f518c112450 con 0x7f518c102690 2024-05-03T20:34:51.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.989+0000 7f51911db700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f518c112750 con 0x7f518c103040 2024-05-03T20:34:51.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.989+0000 7f518b59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f518c1076e0 0x7f518c1a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:51.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.989+0000 7f518b59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f518c1076e0 0x7f518c1a5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59980/0 (socket says 172.21.15.181:59980) 2024-05-03T20:34:51.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.989+0000 7f518b59e700 1 -- 172.21.15.181:0/1663541271 learned_addr learned my addr 172.21.15.181:0/1663541271 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:34:51.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.989+0000 7f518a59c700 1 --2- 172.21.15.181:0/1663541271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f518c103040 0x7f518c19af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:51.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.989+0000 7f518ad9d700 1 --2- 172.21.15.181:0/1663541271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f518c102690 0x7f518c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:51.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.989+0000 7f518b59e700 1 -- 172.21.15.181:0/1663541271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f518c103040 msgr2=0x7f518c19af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:51.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.989+0000 7f518b59e700 1 --2- 172.21.15.181:0/1663541271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f518c103040 0x7f518c19af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:51.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.989+0000 7f518b59e700 1 -- 172.21.15.181:0/1663541271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f518c102690 msgr2=0x7f518c19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:51.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.989+0000 7f518b59e700 1 --2- 172.21.15.181:0/1663541271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f518c102690 0x7f518c19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:51.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.989+0000 7f518b59e700 1 -- 172.21.15.181:0/1663541271 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518c1a5fd0 con 0x7f518c1076e0 2024-05-03T20:34:51.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.989+0000 7f518b59e700 1 --2- 172.21.15.181:0/1663541271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f518c1076e0 0x7f518c1a5a90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f518400b970 tx=0x7f518400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:51.215 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.990+0000 7f5173fff700 1 -- 172.21.15.181:0/1663541271 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184019070 con 0x7f518c1076e0 2024-05-03T20:34:51.215 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.990+0000 7f51911db700 1 -- 172.21.15.181:0/1663541271 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f518c1a61a0 con 0x7f518c1076e0 2024-05-03T20:34:51.215 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.990+0000 7f5173fff700 1 -- 172.21.15.181:0/1663541271 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5184014440 con 0x7f518c1076e0 2024-05-03T20:34:51.215 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.990+0000 7f5173fff700 1 -- 172.21.15.181:0/1663541271 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518400ce60 con 0x7f518c1076e0 2024-05-03T20:34:51.215 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.990+0000 7f51911db700 1 -- 172.21.15.181:0/1663541271 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f518c1a66b0 con 0x7f518c1076e0 2024-05-03T20:34:51.215 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.990+0000 7f51911db700 1 -- 172.21.15.181:0/1663541271 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f518c063bb0 con 0x7f518c1076e0 2024-05-03T20:34:51.215 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.992+0000 7f5173fff700 1 -- 172.21.15.181:0/1663541271 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f518401c070 con 0x7f518c1076e0 2024-05-03T20:34:51.216 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.992+0000 7f5173fff700 1 --2- 172.21.15.181:0/1663541271 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f51740b2910 0x7f51740b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:51.216 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.992+0000 7f518ad9d700 1 --2- 172.21.15.181:0/1663541271 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f51740b2910 0x7f51740b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:51.216 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.992+0000 7f5173fff700 1 -- 172.21.15.181:0/1663541271 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f51840e1f20 con 0x7f518c1076e0 2024-05-03T20:34:51.216 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.993+0000 7f5173fff700 1 --2- 172.21.15.181:0/1663541271 >> [v2:172.21.15.204:6824/488710574,v1:172.21.15.204:6825/488710574] conn(0x7f51740bc450 0x7f51740be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:51.216 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.993+0000 7f5173fff700 1 -- 172.21.15.181:0/1663541271 --> [v2:172.21.15.204:6824/488710574,v1:172.21.15.204:6825/488710574] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5184021e50 con 0x7f51740bc450 2024-05-03T20:34:51.216 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.993+0000 7f5173fff700 1 -- 172.21.15.181:0/1663541271 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f51840e2370 con 0x7f518c1076e0 2024-05-03T20:34:51.216 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.993+0000 7f518ad9d700 1 --2- 172.21.15.181:0/1663541271 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f51740b2910 0x7f51740b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f517c005950 tx=0x7f517c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:51.216 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.993+0000 7f518a59c700 1 --2- 172.21.15.181:0/1663541271 >> [v2:172.21.15.204:6824/488710574,v1:172.21.15.204:6825/488710574] conn(0x7f51740bc450 0x7f51740be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:51.216 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.994+0000 7f518a59c700 1 --2- 172.21.15.181:0/1663541271 >> [v2:172.21.15.204:6824/488710574,v1:172.21.15.204:6825/488710574] conn(0x7f51740bc450 0x7f51740be8f0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:51.217 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:50.994+0000 7f5173fff700 1 -- 172.21.15.181:0/1663541271 <== osd.11 v2:172.21.15.204:6824/488710574 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5184021e50 con 0x7f51740bc450 2024-05-03T20:34:51.217 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.011+0000 7f51911db700 1 -- 172.21.15.181:0/1663541271 --> [v2:172.21.15.204:6824/488710574,v1:172.21.15.204:6825/488710574] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f518c064030 con 0x7f51740bc450 2024-05-03T20:34:51.217 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.012+0000 7f5173fff700 1 -- 172.21.15.181:0/1663541271 <== osd.11 v2:172.21.15.204:6824/488710574 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f518c064030 con 0x7f51740bc450 2024-05-03T20:34:51.217 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.012+0000 7f51911db700 1 -- 172.21.15.181:0/1663541271 >> [v2:172.21.15.204:6824/488710574,v1:172.21.15.204:6825/488710574] conn(0x7f51740bc450 msgr2=0x7f51740be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:51.217 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.012+0000 7f51911db700 1 --2- 172.21.15.181:0/1663541271 >> [v2:172.21.15.204:6824/488710574,v1:172.21.15.204:6825/488710574] conn(0x7f51740bc450 0x7f51740be8f0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:51.217 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.012+0000 7f51911db700 1 -- 172.21.15.181:0/1663541271 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f51740b2910 msgr2=0x7f51740b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:51.217 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.012+0000 7f51911db700 1 --2- 172.21.15.181:0/1663541271 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f51740b2910 0x7f51740b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f517c005950 tx=0x7f517c0058e0 comp rx=0 tx=0).stop 2024-05-03T20:34:51.217 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.012+0000 7f51911db700 1 -- 172.21.15.181:0/1663541271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f518c1076e0 msgr2=0x7f518c1a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:51.217 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.012+0000 7f51911db700 1 --2- 172.21.15.181:0/1663541271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f518c1076e0 0x7f518c1a5a90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f518400b970 tx=0x7f518400bd30 comp rx=0 tx=0).stop 2024-05-03T20:34:51.217 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.012+0000 7f518b59e700 1 -- 172.21.15.181:0/1663541271 reap_dead start 2024-05-03T20:34:51.217 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.012+0000 7f51911db700 1 -- 172.21.15.181:0/1663541271 shutdown_connections 2024-05-03T20:34:51.217 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.012+0000 7f51911db700 1 -- 172.21.15.181:0/1663541271 >> 172.21.15.181:0/1663541271 conn(0x7f518c0783f0 msgr2=0x7f518c100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:51.217 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.013+0000 7f51911db700 1 -- 172.21.15.181:0/1663541271 shutdown_connections 2024-05-03T20:34:51.217 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:51.013+0000 7f51911db700 1 -- 172.21.15.181:0/1663541271 wait complete. 2024-05-03T20:34:51.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:51 smithi181 ceph-mon[29801]: pgmap v321: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-03T20:34:51.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:51 smithi196 ceph-mon[32858]: pgmap v321: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-03T20:34:51.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:51 smithi204 ceph-mon[33595]: pgmap v321: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-03T20:34:52.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.755+0000 7fd488da8700 1 -- 172.21.15.181:0/1333460564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd484107760 msgr2=0x7fd48410f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:52.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.755+0000 7fd488da8700 1 --2- 172.21.15.181:0/1333460564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd484107760 0x7fd48410f2e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00b0d0 tx=0x7fd47c00b3e0 comp rx=0 tx=0).stop 2024-05-03T20:34:52.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.755+0000 7fd488da8700 1 -- 172.21.15.181:0/1333460564 shutdown_connections 2024-05-03T20:34:52.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.755+0000 7fd488da8700 1 --2- 172.21.15.181:0/1333460564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd484107760 0x7fd48410f2e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:52.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.755+0000 7fd488da8700 1 --2- 172.21.15.181:0/1333460564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd484102e00 0x7fd4841031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:52.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.755+0000 7fd488da8700 1 --2- 172.21.15.181:0/1333460564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4841037b0 0x7fd484107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:52.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.755+0000 7fd488da8700 1 -- 172.21.15.181:0/1333460564 >> 172.21.15.181:0/1333460564 conn(0x7fd484078a10 msgr2=0x7fd4840ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:52.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.755+0000 7fd488da8700 1 -- 172.21.15.181:0/1333460564 shutdown_connections 2024-05-03T20:34:52.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.755+0000 7fd488da8700 1 -- 172.21.15.181:0/1333460564 wait complete. 2024-05-03T20:34:52.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.756+0000 7fd488da8700 1 Processor -- start 2024-05-03T20:34:52.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.756+0000 7fd488da8700 1 -- start start 2024-05-03T20:34:52.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.756+0000 7fd488da8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd484102e00 0x7fd4840fde80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:52.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.756+0000 7fd488da8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4841037b0 0x7fd4840fe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:52.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.756+0000 7fd488da8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd484107760 0x7fd4840fe900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:52.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.756+0000 7fd488da8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd484112530 con 0x7fd484107760 2024-05-03T20:34:52.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.756+0000 7fd488da8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd4841123b0 con 0x7fd484102e00 2024-05-03T20:34:52.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.756+0000 7fd488da8700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd4841126b0 con 0x7fd4841037b0 2024-05-03T20:34:52.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.757+0000 7fd482d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd484107760 0x7fd4840fe900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:52.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.757+0000 7fd482d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd484107760 0x7fd4840fe900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60038/0 (socket says 172.21.15.181:60038) 2024-05-03T20:34:52.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.757+0000 7fd48259c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd484102e00 0x7fd4840fde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:52.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.757+0000 7fd482d9d700 1 -- 172.21.15.181:0/1524238650 learned_addr learned my addr 172.21.15.181:0/1524238650 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:34:52.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.757+0000 7fd481d9b700 1 --2- 172.21.15.181:0/1524238650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4841037b0 0x7fd4840fe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:52.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.757+0000 7fd482d9d700 1 -- 172.21.15.181:0/1524238650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4841037b0 msgr2=0x7fd4840fe3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:52.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.757+0000 7fd482d9d700 1 --2- 172.21.15.181:0/1524238650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4841037b0 0x7fd4840fe3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:52.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.757+0000 7fd482d9d700 1 -- 172.21.15.181:0/1524238650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd484102e00 msgr2=0x7fd4840fde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:52.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.757+0000 7fd482d9d700 1 --2- 172.21.15.181:0/1524238650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd484102e00 0x7fd4840fde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:52.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.757+0000 7fd482d9d700 1 -- 172.21.15.181:0/1524238650 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4841a7f20 con 0x7fd484107760 2024-05-03T20:34:52.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.757+0000 7fd48259c700 1 --2- 172.21.15.181:0/1524238650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd484102e00 0x7fd4840fde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T20:34:52.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.757+0000 7fd482d9d700 1 --2- 172.21.15.181:0/1524238650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd484107760 0x7fd4840fe900 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0095c0 tx=0x7fd47c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:52.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.757+0000 7fd4737fe700 1 -- 172.21.15.181:0/1524238650 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c01f070 con 0x7fd484107760 2024-05-03T20:34:52.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.757+0000 7fd488da8700 1 -- 172.21.15.181:0/1524238650 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4841a81b0 con 0x7fd484107760 2024-05-03T20:34:52.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.757+0000 7fd4737fe700 1 -- 172.21.15.181:0/1524238650 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd47c007e20 con 0x7fd484107760 2024-05-03T20:34:52.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.757+0000 7fd488da8700 1 -- 172.21.15.181:0/1524238650 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4841a8660 con 0x7fd484107760 2024-05-03T20:34:52.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.757+0000 7fd4737fe700 1 -- 172.21.15.181:0/1524238650 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c01b4a0 con 0x7fd484107760 2024-05-03T20:34:52.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.758+0000 7fd488da8700 1 -- 172.21.15.181:0/1524238650 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd484063c30 con 0x7fd484107760 2024-05-03T20:34:52.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.760+0000 7fd4737fe700 1 -- 172.21.15.181:0/1524238650 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fd47c022070 con 0x7fd484107760 2024-05-03T20:34:52.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.760+0000 7fd4737fe700 1 --2- 172.21.15.181:0/1524238650 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd46c0b2910 0x7fd46c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:52.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.760+0000 7fd48259c700 1 --2- 172.21.15.181:0/1524238650 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd46c0b2910 0x7fd46c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:52.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.760+0000 7fd4737fe700 1 -- 172.21.15.181:0/1524238650 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7fd47c00f020 con 0x7fd484107760 2024-05-03T20:34:52.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.760+0000 7fd4737fe700 1 --2- 172.21.15.181:0/1524238650 >> [v2:172.21.15.204:6808/855680559,v1:172.21.15.204:6809/855680559] conn(0x7fd46c0bc450 0x7fd46c0be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:52.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.761+0000 7fd4737fe700 1 -- 172.21.15.181:0/1524238650 --> [v2:172.21.15.204:6808/855680559,v1:172.21.15.204:6809/855680559] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd47c031ea0 con 0x7fd46c0bc450 2024-05-03T20:34:52.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.761+0000 7fd4737fe700 1 -- 172.21.15.181:0/1524238650 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fd47c0e3db0 con 0x7fd484107760 2024-05-03T20:34:52.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.761+0000 7fd481d9b700 1 --2- 172.21.15.181:0/1524238650 >> [v2:172.21.15.204:6808/855680559,v1:172.21.15.204:6809/855680559] conn(0x7fd46c0bc450 0x7fd46c0be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:52.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.761+0000 7fd48259c700 1 --2- 172.21.15.181:0/1524238650 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd46c0b2910 0x7fd46c0b4dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd47400afd0 tx=0x7fd47400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:52.767 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.761+0000 7fd481d9b700 1 --2- 172.21.15.181:0/1524238650 >> [v2:172.21.15.204:6808/855680559,v1:172.21.15.204:6809/855680559] conn(0x7fd46c0bc450 0x7fd46c0be8f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:52.767 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.762+0000 7fd4737fe700 1 -- 172.21.15.181:0/1524238650 <== osd.5 v2:172.21.15.204:6808/855680559 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd47c031ea0 con 0x7fd46c0bc450 2024-05-03T20:34:52.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.779+0000 7fd488da8700 1 -- 172.21.15.181:0/1524238650 --> [v2:172.21.15.204:6808/855680559,v1:172.21.15.204:6809/855680559] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd484064060 con 0x7fd46c0bc450 2024-05-03T20:34:52.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.779+0000 7fd4737fe700 1 -- 172.21.15.181:0/1524238650 <== osd.5 v2:172.21.15.204:6808/855680559 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd484064060 con 0x7fd46c0bc450 2024-05-03T20:34:52.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.779+0000 7fd488da8700 1 -- 172.21.15.181:0/1524238650 >> [v2:172.21.15.204:6808/855680559,v1:172.21.15.204:6809/855680559] conn(0x7fd46c0bc450 msgr2=0x7fd46c0be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:52.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.780+0000 7fd488da8700 1 --2- 172.21.15.181:0/1524238650 >> [v2:172.21.15.204:6808/855680559,v1:172.21.15.204:6809/855680559] conn(0x7fd46c0bc450 0x7fd46c0be8f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:52.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.780+0000 7fd488da8700 1 -- 172.21.15.181:0/1524238650 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd46c0b2910 msgr2=0x7fd46c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:52.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.780+0000 7fd488da8700 1 --2- 172.21.15.181:0/1524238650 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd46c0b2910 0x7fd46c0b4dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd47400afd0 tx=0x7fd47400a380 comp rx=0 tx=0).stop 2024-05-03T20:34:52.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.780+0000 7fd488da8700 1 -- 172.21.15.181:0/1524238650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd484107760 msgr2=0x7fd4840fe900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:52.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.780+0000 7fd488da8700 1 --2- 172.21.15.181:0/1524238650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd484107760 0x7fd4840fe900 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0095c0 tx=0x7fd47c007a20 comp rx=0 tx=0).stop 2024-05-03T20:34:52.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.780+0000 7fd482d9d700 1 -- 172.21.15.181:0/1524238650 reap_dead start 2024-05-03T20:34:52.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.780+0000 7fd488da8700 1 -- 172.21.15.181:0/1524238650 shutdown_connections 2024-05-03T20:34:52.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.780+0000 7fd488da8700 1 -- 172.21.15.181:0/1524238650 >> 172.21.15.181:0/1524238650 conn(0x7fd484078a10 msgr2=0x7fd484101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:52.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.780+0000 7fd488da8700 1 -- 172.21.15.181:0/1524238650 shutdown_connections 2024-05-03T20:34:52.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:52.780+0000 7fd488da8700 1 -- 172.21.15.181:0/1524238650 wait complete. 2024-05-03T20:34:53.428 INFO:teuthology.orchestra.run.smithi181.stdout:163208757308 2024-05-03T20:34:53.429 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-03T20:34:53.477 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:53 smithi181 ceph-mon[29801]: pgmap v322: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:34:53.618 INFO:teuthology.orchestra.run.smithi181.stdout:240518168620 2024-05-03T20:34:53.618 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-03T20:34:53.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:53 smithi196 ceph-mon[32858]: pgmap v322: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:34:53.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:53 smithi204 ceph-mon[33595]: pgmap v322: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:34:54.690 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.692+0000 7fdb6b1c1700 1 -- 172.21.15.181:0/3521019936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb64108410 msgr2=0x7fdb641087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:54.690 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.692+0000 7fdb6b1c1700 1 --2- 172.21.15.181:0/3521019936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb64108410 0x7fdb641087f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c00b0a0 tx=0x7fdb5c00b3b0 comp rx=0 tx=0).stop 2024-05-03T20:34:54.690 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.692+0000 7fdb6b1c1700 1 -- 172.21.15.181:0/3521019936 shutdown_connections 2024-05-03T20:34:54.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.692+0000 7fdb6b1c1700 1 --2- 172.21.15.181:0/3521019936 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb64102d60 0x7fdb6410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:54.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.692+0000 7fdb6b1c1700 1 --2- 172.21.15.181:0/3521019936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb64108410 0x7fdb641087f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:54.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.692+0000 7fdb6b1c1700 1 --2- 172.21.15.181:0/3521019936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb641023a0 0x7fdb64102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:54.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.692+0000 7fdb6b1c1700 1 -- 172.21.15.181:0/3521019936 >> 172.21.15.181:0/3521019936 conn(0x7fdb64079350 msgr2=0x7fdb64079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:54.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.692+0000 7fdb6b1c1700 1 -- 172.21.15.181:0/3521019936 shutdown_connections 2024-05-03T20:34:54.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.692+0000 7fdb6b1c1700 1 -- 172.21.15.181:0/3521019936 wait complete. 2024-05-03T20:34:54.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.693+0000 7fdb6b1c1700 1 Processor -- start 2024-05-03T20:34:54.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.693+0000 7fdb6b1c1700 1 -- start start 2024-05-03T20:34:54.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.693+0000 7fdb6b1c1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb641023a0 0x7fdb6419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:54.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.693+0000 7fdb6b1c1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb64102d60 0x7fdb6419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:54.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.693+0000 7fdb6b1c1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb64108410 0x7fdb641989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:54.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.693+0000 7fdb6b1c1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdb64112630 con 0x7fdb64102d60 2024-05-03T20:34:54.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.693+0000 7fdb6b1c1700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb641124b0 con 0x7fdb64108410 2024-05-03T20:34:54.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.693+0000 7fdb6b1c1700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdb641127b0 con 0x7fdb641023a0 2024-05-03T20:34:54.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.693+0000 7fdb63fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb64102d60 0x7fdb6419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:54.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.693+0000 7fdb63fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb64102d60 0x7fdb6419ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60070/0 (socket says 172.21.15.181:60070) 2024-05-03T20:34:54.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.693+0000 7fdb63fff700 1 -- 172.21.15.181:0/4143782959 learned_addr learned my addr 172.21.15.181:0/4143782959 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:34:54.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.693+0000 7fdb68f5d700 1 --2- 172.21.15.181:0/4143782959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb641023a0 0x7fdb6419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:54.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.693+0000 7fdb6975e700 1 --2- 172.21.15.181:0/4143782959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb64108410 0x7fdb641989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:54.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.693+0000 7fdb63fff700 1 -- 172.21.15.181:0/4143782959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb641023a0 msgr2=0x7fdb6419e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:54.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.693+0000 7fdb63fff700 1 --2- 172.21.15.181:0/4143782959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb641023a0 0x7fdb6419e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:54.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.693+0000 7fdb63fff700 1 -- 172.21.15.181:0/4143782959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb64108410 msgr2=0x7fdb641989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:54.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.693+0000 7fdb63fff700 1 --2- 172.21.15.181:0/4143782959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb64108410 0x7fdb641989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:54.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.693+0000 7fdb63fff700 1 -- 172.21.15.181:0/4143782959 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb64199150 con 0x7fdb64102d60 2024-05-03T20:34:54.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.693+0000 7fdb63fff700 1 --2- 172.21.15.181:0/4143782959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb64102d60 0x7fdb6419ed70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800b530 tx=0x7fdb5800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:54.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.694+0000 7fdb61ffb700 1 -- 172.21.15.181:0/4143782959 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb58015020 con 0x7fdb64102d60 2024-05-03T20:34:54.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.694+0000 7fdb61ffb700 1 -- 172.21.15.181:0/4143782959 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb58011ce0 con 0x7fdb64102d60 2024-05-03T20:34:54.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.694+0000 7fdb61ffb700 1 -- 172.21.15.181:0/4143782959 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb58010e40 con 0x7fdb64102d60 2024-05-03T20:34:54.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.694+0000 7fdb6b1c1700 1 -- 172.21.15.181:0/4143782959 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb64199440 con 0x7fdb64102d60 2024-05-03T20:34:54.695 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.694+0000 7fdb6b1c1700 1 -- 172.21.15.181:0/4143782959 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb641a5700 con 0x7fdb64102d60 2024-05-03T20:34:54.695 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.695+0000 7fdb6b1c1700 1 -- 172.21.15.181:0/4143782959 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdb64063bf0 con 0x7fdb64102d60 2024-05-03T20:34:54.696 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.696+0000 7fdb61ffb700 1 -- 172.21.15.181:0/4143782959 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fdb58018020 con 0x7fdb64102d60 2024-05-03T20:34:54.696 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.697+0000 7fdb61ffb700 1 --2- 172.21.15.181:0/4143782959 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fdb4c0b2910 0x7fdb4c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:54.696 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.697+0000 7fdb68f5d700 1 --2- 172.21.15.181:0/4143782959 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fdb4c0b2910 0x7fdb4c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:54.696 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.697+0000 7fdb61ffb700 1 -- 172.21.15.181:0/4143782959 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7fdb580e2240 con 0x7fdb64102d60 2024-05-03T20:34:54.696 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.697+0000 7fdb61ffb700 1 --2- 172.21.15.181:0/4143782959 >> [v2:172.21.15.196:6824/3980645995,v1:172.21.15.196:6825/3980645995] conn(0x7fdb4c0bc450 0x7fdb4c0be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:54.696 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.697+0000 7fdb61ffb700 1 -- 172.21.15.181:0/4143782959 --> [v2:172.21.15.196:6824/3980645995,v1:172.21.15.196:6825/3980645995] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb5801be50 con 0x7fdb4c0bc450 2024-05-03T20:34:54.696 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.697+0000 7fdb61ffb700 1 -- 172.21.15.181:0/4143782959 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fdb580e2690 con 0x7fdb64102d60 2024-05-03T20:34:54.696 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.697+0000 7fdb68f5d700 1 --2- 172.21.15.181:0/4143782959 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fdb4c0b2910 0x7fdb4c0b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c009fd0 tx=0x7fdb5c00b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:54.696 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.697+0000 7fdb6975e700 1 --2- 172.21.15.181:0/4143782959 >> [v2:172.21.15.196:6824/3980645995,v1:172.21.15.196:6825/3980645995] conn(0x7fdb4c0bc450 0x7fdb4c0be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:54.696 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.698+0000 7fdb6975e700 1 --2- 172.21.15.181:0/4143782959 >> [v2:172.21.15.196:6824/3980645995,v1:172.21.15.196:6825/3980645995] conn(0x7fdb4c0bc450 0x7fdb4c0be8f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:54.697 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.699+0000 7fdb61ffb700 1 -- 172.21.15.181:0/4143782959 <== osd.10 v2:172.21.15.196:6824/3980645995 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdb5801be50 con 0x7fdb4c0bc450 2024-05-03T20:34:54.713 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.715+0000 7fdb6b1c1700 1 -- 172.21.15.181:0/4143782959 --> [v2:172.21.15.196:6824/3980645995,v1:172.21.15.196:6825/3980645995] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdb64064070 con 0x7fdb4c0bc450 2024-05-03T20:34:54.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.715+0000 7fdb61ffb700 1 -- 172.21.15.181:0/4143782959 <== osd.10 v2:172.21.15.196:6824/3980645995 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdb64064070 con 0x7fdb4c0bc450 2024-05-03T20:34:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.716+0000 7fdb6b1c1700 1 -- 172.21.15.181:0/4143782959 >> [v2:172.21.15.196:6824/3980645995,v1:172.21.15.196:6825/3980645995] conn(0x7fdb4c0bc450 msgr2=0x7fdb4c0be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.716+0000 7fdb6b1c1700 1 --2- 172.21.15.181:0/4143782959 >> [v2:172.21.15.196:6824/3980645995,v1:172.21.15.196:6825/3980645995] conn(0x7fdb4c0bc450 0x7fdb4c0be8f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.716+0000 7fdb6b1c1700 1 -- 172.21.15.181:0/4143782959 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fdb4c0b2910 msgr2=0x7fdb4c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.716+0000 7fdb6b1c1700 1 --2- 172.21.15.181:0/4143782959 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fdb4c0b2910 0x7fdb4c0b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c009fd0 tx=0x7fdb5c00b7b0 comp rx=0 tx=0).stop 2024-05-03T20:34:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.716+0000 7fdb6b1c1700 1 -- 172.21.15.181:0/4143782959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb64102d60 msgr2=0x7fdb6419ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.716+0000 7fdb6b1c1700 1 --2- 172.21.15.181:0/4143782959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb64102d60 0x7fdb6419ed70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800b530 tx=0x7fdb5800b840 comp rx=0 tx=0).stop 2024-05-03T20:34:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.716+0000 7fdb6975e700 1 -- 172.21.15.181:0/4143782959 reap_dead start 2024-05-03T20:34:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.716+0000 7fdb6b1c1700 1 -- 172.21.15.181:0/4143782959 shutdown_connections 2024-05-03T20:34:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.716+0000 7fdb6b1c1700 1 -- 172.21.15.181:0/4143782959 >> 172.21.15.181:0/4143782959 conn(0x7fdb64079350 msgr2=0x7fdb641012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.716+0000 7fdb6b1c1700 1 -- 172.21.15.181:0/4143782959 shutdown_connections 2024-05-03T20:34:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:54.716+0000 7fdb6b1c1700 1 -- 172.21.15.181:0/4143782959 wait complete. 2024-05-03T20:34:55.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:55 smithi181 ceph-mon[29801]: pgmap v323: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T20:34:55.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:55 smithi204 ceph-mon[33595]: pgmap v323: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T20:34:55.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:55 smithi196 ceph-mon[32858]: pgmap v323: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T20:34:56.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.801+0000 7f84333c5700 1 -- 172.21.15.181:0/1496110353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f842c108410 msgr2=0x7f842c1087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:56.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.801+0000 7f84333c5700 1 --2- 172.21.15.181:0/1496110353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f842c108410 0x7f842c1087f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8428009990 tx=0x7f8428009ca0 comp rx=0 tx=0).stop 2024-05-03T20:34:56.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.801+0000 7f84333c5700 1 -- 172.21.15.181:0/1496110353 shutdown_connections 2024-05-03T20:34:56.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.801+0000 7f84333c5700 1 --2- 172.21.15.181:0/1496110353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f842c102d60 0x7f842c10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:56.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.801+0000 7f84333c5700 1 --2- 172.21.15.181:0/1496110353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f842c108410 0x7f842c1087f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:56.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.801+0000 7f84333c5700 1 --2- 172.21.15.181:0/1496110353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f842c1023a0 0x7f842c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:56.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.801+0000 7f84333c5700 1 -- 172.21.15.181:0/1496110353 >> 172.21.15.181:0/1496110353 conn(0x7f842c079350 msgr2=0x7f842c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:56.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.801+0000 7f84333c5700 1 -- 172.21.15.181:0/1496110353 shutdown_connections 2024-05-03T20:34:56.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.801+0000 7f84333c5700 1 -- 172.21.15.181:0/1496110353 wait complete. 2024-05-03T20:34:56.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.802+0000 7f84333c5700 1 Processor -- start 2024-05-03T20:34:56.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.802+0000 7f84333c5700 1 -- start start 2024-05-03T20:34:56.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.802+0000 7f84333c5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f842c1023a0 0x7f842c104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:56.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.802+0000 7f84333c5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f842c102d60 0x7f842c1045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:56.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.802+0000 7f84333c5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f842c108410 0x7f842c104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:56.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.802+0000 7f84333c5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f842c112530 con 0x7f842c102d60 2024-05-03T20:34:56.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.802+0000 7f84333c5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f842c1123b0 con 0x7f842c1023a0 2024-05-03T20:34:56.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.802+0000 7f84333c5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f842c1126b0 con 0x7f842c108410 2024-05-03T20:34:56.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.802+0000 7f8430960700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f842c102d60 0x7f842c1045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:56.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.802+0000 7f8430960700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f842c102d60 0x7f842c1045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60048/0 (socket says 172.21.15.181:60048) 2024-05-03T20:34:56.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.802+0000 7f8430960700 1 -- 172.21.15.181:0/1587882455 learned_addr learned my addr 172.21.15.181:0/1587882455 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:34:56.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.802+0000 7f8431962700 1 --2- 172.21.15.181:0/1587882455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f842c108410 0x7f842c104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:56.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.802+0000 7f8431161700 1 --2- 172.21.15.181:0/1587882455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f842c1023a0 0x7f842c104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:56.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.802+0000 7f8430960700 1 -- 172.21.15.181:0/1587882455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f842c108410 msgr2=0x7f842c104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:56.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.802+0000 7f8430960700 1 --2- 172.21.15.181:0/1587882455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f842c108410 0x7f842c104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:56.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.802+0000 7f8430960700 1 -- 172.21.15.181:0/1587882455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f842c1023a0 msgr2=0x7f842c104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:56.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.802+0000 7f8430960700 1 --2- 172.21.15.181:0/1587882455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f842c1023a0 0x7f842c104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:56.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.802+0000 7f8430960700 1 -- 172.21.15.181:0/1587882455 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842c1a80a0 con 0x7f842c102d60 2024-05-03T20:34:56.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.803+0000 7f8430960700 1 --2- 172.21.15.181:0/1587882455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f842c102d60 0x7f842c1045c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f841c00d950 tx=0x7f841c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:56.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.803+0000 7f84227fc700 1 -- 172.21.15.181:0/1587882455 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c016070 con 0x7f842c102d60 2024-05-03T20:34:56.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.803+0000 7f84333c5700 1 -- 172.21.15.181:0/1587882455 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f842c1a8390 con 0x7f842c102d60 2024-05-03T20:34:56.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.803+0000 7f84227fc700 1 -- 172.21.15.181:0/1587882455 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f841c00fd40 con 0x7f842c102d60 2024-05-03T20:34:56.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.803+0000 7f84227fc700 1 -- 172.21.15.181:0/1587882455 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c00e3b0 con 0x7f842c102d60 2024-05-03T20:34:56.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.804+0000 7f84333c5700 1 -- 172.21.15.181:0/1587882455 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f842c1a8920 con 0x7f842c102d60 2024-05-03T20:34:56.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.804+0000 7f84333c5700 1 -- 172.21.15.181:0/1587882455 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f842c1074a0 con 0x7f842c102d60 2024-05-03T20:34:56.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.806+0000 7f84227fc700 1 -- 172.21.15.181:0/1587882455 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f841c019070 con 0x7f842c102d60 2024-05-03T20:34:56.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.806+0000 7f84227fc700 1 --2- 172.21.15.181:0/1587882455 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f84180b2910 0x7f84180b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:56.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.806+0000 7f8431161700 1 --2- 172.21.15.181:0/1587882455 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f84180b2910 0x7f84180b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:56.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.806+0000 7f84227fc700 1 -- 172.21.15.181:0/1587882455 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f841c0e18a0 con 0x7f842c102d60 2024-05-03T20:34:56.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.806+0000 7f84227fc700 1 --2- 172.21.15.181:0/1587882455 >> [v2:172.21.15.181:6826/241693472,v1:172.21.15.181:6827/241693472] conn(0x7f84180bc450 0x7f84180be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:56.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.806+0000 7f84227fc700 1 -- 172.21.15.181:0/1587882455 --> [v2:172.21.15.181:6826/241693472,v1:172.21.15.181:6827/241693472] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f841c01aec0 con 0x7f84180bc450 2024-05-03T20:34:56.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.806+0000 7f84227fc700 1 -- 172.21.15.181:0/1587882455 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f841c0e62c0 con 0x7f842c102d60 2024-05-03T20:34:56.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.807+0000 7f8431161700 1 --2- 172.21.15.181:0/1587882455 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f84180b2910 0x7f84180b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f84280098e0 tx=0x7f8428005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:56.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.807+0000 7f8431962700 1 --2- 172.21.15.181:0/1587882455 >> [v2:172.21.15.181:6826/241693472,v1:172.21.15.181:6827/241693472] conn(0x7f84180bc450 0x7f84180be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:56.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.807+0000 7f8431962700 1 --2- 172.21.15.181:0/1587882455 >> [v2:172.21.15.181:6826/241693472,v1:172.21.15.181:6827/241693472] conn(0x7f84180bc450 0x7f84180be8f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:56.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.808+0000 7f84227fc700 1 -- 172.21.15.181:0/1587882455 <== osd.9 v2:172.21.15.181:6826/241693472 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f841c01aec0 con 0x7f84180bc450 2024-05-03T20:34:56.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.824+0000 7f84333c5700 1 -- 172.21.15.181:0/1587882455 --> [v2:172.21.15.181:6826/241693472,v1:172.21.15.181:6827/241693472] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f842c103ac0 con 0x7f84180bc450 2024-05-03T20:34:56.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.825+0000 7f84227fc700 1 -- 172.21.15.181:0/1587882455 <== osd.9 v2:172.21.15.181:6826/241693472 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f842c103ac0 con 0x7f84180bc450 2024-05-03T20:34:56.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.825+0000 7f84333c5700 1 -- 172.21.15.181:0/1587882455 >> [v2:172.21.15.181:6826/241693472,v1:172.21.15.181:6827/241693472] conn(0x7f84180bc450 msgr2=0x7f84180be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:56.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.825+0000 7f84333c5700 1 --2- 172.21.15.181:0/1587882455 >> [v2:172.21.15.181:6826/241693472,v1:172.21.15.181:6827/241693472] conn(0x7f84180bc450 0x7f84180be8f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:56.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.825+0000 7f84333c5700 1 -- 172.21.15.181:0/1587882455 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f84180b2910 msgr2=0x7f84180b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:56.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.825+0000 7f84333c5700 1 --2- 172.21.15.181:0/1587882455 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f84180b2910 0x7f84180b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f84280098e0 tx=0x7f8428005960 comp rx=0 tx=0).stop 2024-05-03T20:34:56.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.825+0000 7f84333c5700 1 -- 172.21.15.181:0/1587882455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f842c102d60 msgr2=0x7f842c1045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:56.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.825+0000 7f84333c5700 1 --2- 172.21.15.181:0/1587882455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f842c102d60 0x7f842c1045c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f841c00d950 tx=0x7f841c00dc60 comp rx=0 tx=0).stop 2024-05-03T20:34:56.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.825+0000 7f8431962700 1 -- 172.21.15.181:0/1587882455 reap_dead start 2024-05-03T20:34:56.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.825+0000 7f84333c5700 1 -- 172.21.15.181:0/1587882455 shutdown_connections 2024-05-03T20:34:56.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.825+0000 7f84333c5700 1 -- 172.21.15.181:0/1587882455 >> 172.21.15.181:0/1587882455 conn(0x7f842c079350 msgr2=0x7f842c06b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:56.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.825+0000 7f84333c5700 1 -- 172.21.15.181:0/1587882455 shutdown_connections 2024-05-03T20:34:56.133 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:53.825+0000 7f84333c5700 1 -- 172.21.15.181:0/1587882455 wait complete. 2024-05-03T20:34:56.754 INFO:teuthology.orchestra.run.smithi181.stdout:266287972393 2024-05-03T20:34:56.755 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-03T20:34:57.077 INFO:teuthology.orchestra.run.smithi181.stdout:141733920830 2024-05-03T20:34:57.077 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-03T20:34:57.621 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:57 smithi181 ceph-mon[29801]: pgmap v324: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-05-03T20:34:57.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:57 smithi196 ceph-mon[32858]: pgmap v324: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-05-03T20:34:57.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:57 smithi204 ceph-mon[33595]: pgmap v324: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-05-03T20:34:57.939 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.939+0000 7f590ad75700 1 -- 172.21.15.181:0/1403825928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59041029e0 msgr2=0x7f5904102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:57.939 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.939+0000 7f590ad75700 1 --2- 172.21.15.181:0/1403825928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59041029e0 0x7f5904102dc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f58f80099c0 tx=0x7f58f8009cd0 comp rx=0 tx=0).stop 2024-05-03T20:34:57.940 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.939+0000 7f590ad75700 1 -- 172.21.15.181:0/1403825928 shutdown_connections 2024-05-03T20:34:57.940 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.939+0000 7f590ad75700 1 --2- 172.21.15.181:0/1403825928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5904107730 0x7f590410f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:57.940 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.939+0000 7f590ad75700 1 --2- 172.21.15.181:0/1403825928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59041029e0 0x7f5904102dc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:57.940 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.939+0000 7f590ad75700 1 --2- 172.21.15.181:0/1403825928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5904103390 0x7f5904107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:57.940 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.939+0000 7f590ad75700 1 -- 172.21.15.181:0/1403825928 >> 172.21.15.181:0/1403825928 conn(0x7f5904079350 msgr2=0x7f5904079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:57.940 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.939+0000 7f590ad75700 1 -- 172.21.15.181:0/1403825928 shutdown_connections 2024-05-03T20:34:57.940 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.939+0000 7f590ad75700 1 -- 172.21.15.181:0/1403825928 wait complete. 2024-05-03T20:34:57.940 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.939+0000 7f590ad75700 1 Processor -- start 2024-05-03T20:34:57.940 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.939+0000 7f590ad75700 1 -- start start 2024-05-03T20:34:57.940 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.940+0000 7f590ad75700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59041029e0 0x7f590419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:57.940 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.940+0000 7f590ad75700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5904103390 0x7f590419b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:57.940 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.940+0000 7f590ad75700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5904107730 0x7f59041a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:57.940 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.940+0000 7f590ad75700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5904112670 con 0x7f5904103390 2024-05-03T20:34:57.940 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.940+0000 7f590ad75700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f59041124f0 con 0x7f59041029e0 2024-05-03T20:34:57.940 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.940+0000 7f590ad75700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f59041127f0 con 0x7f5904107730 2024-05-03T20:34:57.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.940+0000 7f5903fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5904103390 0x7f590419b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:57.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.940+0000 7f5903fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5904103390 0x7f590419b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50700/0 (socket says 172.21.15.181:50700) 2024-05-03T20:34:57.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.940+0000 7f5903fff700 1 -- 172.21.15.181:0/640168497 learned_addr learned my addr 172.21.15.181:0/640168497 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:34:57.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.940+0000 7f5909312700 1 --2- 172.21.15.181:0/640168497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5904107730 0x7f59041a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:57.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.940+0000 7f5908b11700 1 --2- 172.21.15.181:0/640168497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59041029e0 0x7f590419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:57.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.940+0000 7f5903fff700 1 -- 172.21.15.181:0/640168497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5904107730 msgr2=0x7f59041a5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:57.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.940+0000 7f5903fff700 1 --2- 172.21.15.181:0/640168497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5904107730 0x7f59041a5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:57.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.940+0000 7f5903fff700 1 -- 172.21.15.181:0/640168497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59041029e0 msgr2=0x7f590419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:57.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.940+0000 7f5903fff700 1 --2- 172.21.15.181:0/640168497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59041029e0 0x7f590419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:57.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.940+0000 7f5903fff700 1 -- 172.21.15.181:0/640168497 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59041a6170 con 0x7f5904103390 2024-05-03T20:34:57.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.940+0000 7f5908b11700 1 --2- 172.21.15.181:0/640168497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59041029e0 0x7f590419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:34:57.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.940+0000 7f5903fff700 1 --2- 172.21.15.181:0/640168497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5904103390 0x7f590419b0f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f58f000b4d0 tx=0x7f58f000b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:57.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.941+0000 7f5901ffb700 1 -- 172.21.15.181:0/640168497 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0015020 con 0x7f5904103390 2024-05-03T20:34:57.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.941+0000 7f5901ffb700 1 -- 172.21.15.181:0/640168497 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58f0011ce0 con 0x7f5904103390 2024-05-03T20:34:57.942 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.941+0000 7f590ad75700 1 -- 172.21.15.181:0/640168497 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59041a6340 con 0x7f5904103390 2024-05-03T20:34:57.942 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.941+0000 7f5901ffb700 1 -- 172.21.15.181:0/640168497 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0010cd0 con 0x7f5904103390 2024-05-03T20:34:57.942 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.941+0000 7f590ad75700 1 -- 172.21.15.181:0/640168497 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59041a6800 con 0x7f5904103390 2024-05-03T20:34:57.942 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.942+0000 7f590ad75700 1 -- 172.21.15.181:0/640168497 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5904063bb0 con 0x7f5904103390 2024-05-03T20:34:57.942 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.944+0000 7f5901ffb700 1 -- 172.21.15.181:0/640168497 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f58f0018020 con 0x7f5904103390 2024-05-03T20:34:57.943 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.944+0000 7f5901ffb700 1 --2- 172.21.15.181:0/640168497 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f58f40b2910 0x7f58f40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:57.943 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.945+0000 7f5908b11700 1 --2- 172.21.15.181:0/640168497 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f58f40b2910 0x7f58f40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:57.943 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.945+0000 7f5901ffb700 1 -- 172.21.15.181:0/640168497 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f58f00e2140 con 0x7f5904103390 2024-05-03T20:34:57.943 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.945+0000 7f5901ffb700 1 --2- 172.21.15.181:0/640168497 >> [v2:172.21.15.196:6816/824890163,v1:172.21.15.196:6817/824890163] conn(0x7f58f40bc450 0x7f58f40be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:57.944 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.945+0000 7f5901ffb700 1 -- 172.21.15.181:0/640168497 --> [v2:172.21.15.196:6816/824890163,v1:172.21.15.196:6817/824890163] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f58f001be50 con 0x7f58f40bc450 2024-05-03T20:34:57.944 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.945+0000 7f5901ffb700 1 -- 172.21.15.181:0/640168497 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f58f00e2590 con 0x7f5904103390 2024-05-03T20:34:57.944 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.945+0000 7f5908b11700 1 --2- 172.21.15.181:0/640168497 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f58f40b2910 0x7f58f40b4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f58f8000c00 tx=0x7f58f8009f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:57.944 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.945+0000 7f5909312700 1 --2- 172.21.15.181:0/640168497 >> [v2:172.21.15.196:6816/824890163,v1:172.21.15.196:6817/824890163] conn(0x7f58f40bc450 0x7f58f40be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:57.944 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.946+0000 7f5909312700 1 --2- 172.21.15.181:0/640168497 >> [v2:172.21.15.196:6816/824890163,v1:172.21.15.196:6817/824890163] conn(0x7f58f40bc450 0x7f58f40be8f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:57.945 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.946+0000 7f5901ffb700 1 -- 172.21.15.181:0/640168497 <== osd.7 v2:172.21.15.196:6816/824890163 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f58f001be50 con 0x7f58f40bc450 2024-05-03T20:34:57.962 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.964+0000 7f590ad75700 1 -- 172.21.15.181:0/640168497 --> [v2:172.21.15.196:6816/824890163,v1:172.21.15.196:6817/824890163] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5904063fb0 con 0x7f58f40bc450 2024-05-03T20:34:57.963 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.965+0000 7f5901ffb700 1 -- 172.21.15.181:0/640168497 <== osd.7 v2:172.21.15.196:6816/824890163 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5904063fb0 con 0x7f58f40bc450 2024-05-03T20:34:57.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.965+0000 7f590ad75700 1 -- 172.21.15.181:0/640168497 >> [v2:172.21.15.196:6816/824890163,v1:172.21.15.196:6817/824890163] conn(0x7f58f40bc450 msgr2=0x7f58f40be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:57.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.965+0000 7f590ad75700 1 --2- 172.21.15.181:0/640168497 >> [v2:172.21.15.196:6816/824890163,v1:172.21.15.196:6817/824890163] conn(0x7f58f40bc450 0x7f58f40be8f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:57.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.965+0000 7f590ad75700 1 -- 172.21.15.181:0/640168497 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f58f40b2910 msgr2=0x7f58f40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:57.965 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.965+0000 7f590ad75700 1 --2- 172.21.15.181:0/640168497 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f58f40b2910 0x7f58f40b4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f58f8000c00 tx=0x7f58f8009f90 comp rx=0 tx=0).stop 2024-05-03T20:34:57.965 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.965+0000 7f590ad75700 1 -- 172.21.15.181:0/640168497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5904103390 msgr2=0x7f590419b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:57.965 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.965+0000 7f590ad75700 1 --2- 172.21.15.181:0/640168497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5904103390 0x7f590419b0f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f58f000b4d0 tx=0x7f58f000b7e0 comp rx=0 tx=0).stop 2024-05-03T20:34:57.965 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.965+0000 7f5909312700 1 -- 172.21.15.181:0/640168497 reap_dead start 2024-05-03T20:34:57.965 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.966+0000 7f590ad75700 1 -- 172.21.15.181:0/640168497 shutdown_connections 2024-05-03T20:34:57.965 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.966+0000 7f590ad75700 1 -- 172.21.15.181:0/640168497 >> 172.21.15.181:0/640168497 conn(0x7f5904079350 msgr2=0x7f5904100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:57.965 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.966+0000 7f590ad75700 1 -- 172.21.15.181:0/640168497 shutdown_connections 2024-05-03T20:34:57.965 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:57.966+0000 7f590ad75700 1 -- 172.21.15.181:0/640168497 wait complete. 2024-05-03T20:34:58.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.176+0000 7f5bff887700 1 -- 172.21.15.181:0/1592135637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf8104160 msgr2=0x7f5bf8108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:58.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.176+0000 7f5bff887700 1 --2- 172.21.15.181:0/1592135637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf8104160 0x7f5bf8108150 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5be80099c0 tx=0x7f5be8009cd0 comp rx=0 tx=0).stop 2024-05-03T20:34:58.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.176+0000 7f5bff887700 1 -- 172.21.15.181:0/1592135637 shutdown_connections 2024-05-03T20:34:58.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.176+0000 7f5bff887700 1 --2- 172.21.15.181:0/1592135637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bf8108880 0x7f5bf8110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:58.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.176+0000 7f5bff887700 1 --2- 172.21.15.181:0/1592135637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5bf81037b0 0x7f5bf8103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:58.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.176+0000 7f5bff887700 1 --2- 172.21.15.181:0/1592135637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf8104160 0x7f5bf8108150 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:58.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.176+0000 7f5bff887700 1 -- 172.21.15.181:0/1592135637 >> 172.21.15.181:0/1592135637 conn(0x7f5bf807a580 msgr2=0x7f5bf807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:58.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.177+0000 7f5bff887700 1 -- 172.21.15.181:0/1592135637 shutdown_connections 2024-05-03T20:34:58.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.177+0000 7f5bff887700 1 -- 172.21.15.181:0/1592135637 wait complete. 2024-05-03T20:34:58.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.177+0000 7f5bff887700 1 Processor -- start 2024-05-03T20:34:58.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.177+0000 7f5bff887700 1 -- start start 2024-05-03T20:34:58.178 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.178+0000 7f5bff887700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bf81037b0 0x7f5bf810f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:58.178 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.178+0000 7f5bff887700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf8104160 0x7f5bf810fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:58.178 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.178+0000 7f5bff887700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5bf8108880 0x7f5bf81a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:58.178 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.178+0000 7f5bff887700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf8113660 con 0x7f5bf8104160 2024-05-03T20:34:58.178 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.178+0000 7f5bff887700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf81134e0 con 0x7f5bf8108880 2024-05-03T20:34:58.178 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.178+0000 7f5bff887700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf81137e0 con 0x7f5bf81037b0 2024-05-03T20:34:58.178 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.178+0000 7f5bfce22700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf8104160 0x7f5bf810fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:58.178 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.178+0000 7f5bfce22700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf8104160 0x7f5bf810fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50706/0 (socket says 172.21.15.181:50706) 2024-05-03T20:34:58.178 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.178+0000 7f5bfce22700 1 -- 172.21.15.181:0/3679392933 learned_addr learned my addr 172.21.15.181:0/3679392933 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:34:58.178 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.178+0000 7f5bfde24700 1 --2- 172.21.15.181:0/3679392933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5bf8108880 0x7f5bf81a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:58.178 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.178+0000 7f5bfd623700 1 --2- 172.21.15.181:0/3679392933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bf81037b0 0x7f5bf810f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:58.178 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.178+0000 7f5bfce22700 1 -- 172.21.15.181:0/3679392933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bf81037b0 msgr2=0x7f5bf810f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:58.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.178+0000 7f5bfce22700 1 --2- 172.21.15.181:0/3679392933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bf81037b0 0x7f5bf810f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:58.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.178+0000 7f5bfce22700 1 -- 172.21.15.181:0/3679392933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5bf8108880 msgr2=0x7f5bf81a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:58.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.178+0000 7f5bfce22700 1 --2- 172.21.15.181:0/3679392933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5bf8108880 0x7f5bf81a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:58.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.178+0000 7f5bfce22700 1 -- 172.21.15.181:0/3679392933 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf81a5060 con 0x7f5bf8104160 2024-05-03T20:34:58.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.179+0000 7f5bfd623700 1 --2- 172.21.15.181:0/3679392933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bf81037b0 0x7f5bf810f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:34:58.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.179+0000 7f5bfce22700 1 --2- 172.21.15.181:0/3679392933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf8104160 0x7f5bf810fac0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5be8000c00 tx=0x7f5be80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:58.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.179+0000 7f5bee7fc700 1 -- 172.21.15.181:0/3679392933 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be8027070 con 0x7f5bf8104160 2024-05-03T20:34:58.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.179+0000 7f5bff887700 1 -- 172.21.15.181:0/3679392933 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf81a52f0 con 0x7f5bf8104160 2024-05-03T20:34:58.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.179+0000 7f5bee7fc700 1 -- 172.21.15.181:0/3679392933 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5be80040e0 con 0x7f5bf8104160 2024-05-03T20:34:58.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.179+0000 7f5bff887700 1 -- 172.21.15.181:0/3679392933 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf81acbd0 con 0x7f5bf8104160 2024-05-03T20:34:58.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.179+0000 7f5bee7fc700 1 -- 172.21.15.181:0/3679392933 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be8022e30 con 0x7f5bf8104160 2024-05-03T20:34:58.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.181+0000 7f5be3fff700 1 -- 172.21.15.181:0/3679392933 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5bf8063c30 con 0x7f5bf8104160 2024-05-03T20:34:58.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.183+0000 7f5bee7fc700 1 -- 172.21.15.181:0/3679392933 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f5be802a070 con 0x7f5bf8104160 2024-05-03T20:34:58.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.184+0000 7f5bee7fc700 1 --2- 172.21.15.181:0/3679392933 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5be40b2990 0x7f5be40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:58.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.184+0000 7f5bfd623700 1 --2- 172.21.15.181:0/3679392933 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5be40b2990 0x7f5be40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:58.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.184+0000 7f5bee7fc700 1 -- 172.21.15.181:0/3679392933 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f5be80e3100 con 0x7f5bf8104160 2024-05-03T20:34:58.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.184+0000 7f5bee7fc700 1 --2- 172.21.15.181:0/3679392933 >> [v2:172.21.15.196:6800/2136873375,v1:172.21.15.196:6801/2136873375] conn(0x7f5be40bc4d0 0x7f5be40be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:34:58.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.184+0000 7f5bee7fc700 1 -- 172.21.15.181:0/3679392933 --> [v2:172.21.15.196:6800/2136873375,v1:172.21.15.196:6801/2136873375] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5be800eea0 con 0x7f5be40bc4d0 2024-05-03T20:34:58.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.184+0000 7f5bee7fc700 1 -- 172.21.15.181:0/3679392933 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f5be80ea050 con 0x7f5bf8104160 2024-05-03T20:34:58.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.184+0000 7f5bfde24700 1 --2- 172.21.15.181:0/3679392933 >> [v2:172.21.15.196:6800/2136873375,v1:172.21.15.196:6801/2136873375] conn(0x7f5be40bc4d0 0x7f5be40be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:34:58.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.184+0000 7f5bfd623700 1 --2- 172.21.15.181:0/3679392933 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5be40b2990 0x7f5be40b4e50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4005fd0 tx=0x7f5bf4005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:58.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.185+0000 7f5bfde24700 1 --2- 172.21.15.181:0/3679392933 >> [v2:172.21.15.196:6800/2136873375,v1:172.21.15.196:6801/2136873375] conn(0x7f5be40bc4d0 0x7f5be40be970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:34:58.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.186+0000 7f5bee7fc700 1 -- 172.21.15.181:0/3679392933 <== osd.1 v2:172.21.15.196:6800/2136873375 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5be800eea0 con 0x7f5be40bc4d0 2024-05-03T20:34:58.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.206+0000 7f5be3fff700 1 -- 172.21.15.181:0/3679392933 --> [v2:172.21.15.196:6800/2136873375,v1:172.21.15.196:6801/2136873375] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5bf8064060 con 0x7f5be40bc4d0 2024-05-03T20:34:58.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.207+0000 7f5bee7fc700 1 -- 172.21.15.181:0/3679392933 <== osd.1 v2:172.21.15.196:6800/2136873375 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5bf8064060 con 0x7f5be40bc4d0 2024-05-03T20:34:58.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.207+0000 7f5be3fff700 1 -- 172.21.15.181:0/3679392933 >> [v2:172.21.15.196:6800/2136873375,v1:172.21.15.196:6801/2136873375] conn(0x7f5be40bc4d0 msgr2=0x7f5be40be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:58.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.207+0000 7f5be3fff700 1 --2- 172.21.15.181:0/3679392933 >> [v2:172.21.15.196:6800/2136873375,v1:172.21.15.196:6801/2136873375] conn(0x7f5be40bc4d0 0x7f5be40be970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:34:58.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.208+0000 7f5be3fff700 1 -- 172.21.15.181:0/3679392933 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5be40b2990 msgr2=0x7f5be40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:58.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.208+0000 7f5be3fff700 1 --2- 172.21.15.181:0/3679392933 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5be40b2990 0x7f5be40b4e50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4005fd0 tx=0x7f5bf4005e20 comp rx=0 tx=0).stop 2024-05-03T20:34:58.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.208+0000 7f5be3fff700 1 -- 172.21.15.181:0/3679392933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf8104160 msgr2=0x7f5bf810fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:34:58.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.208+0000 7f5be3fff700 1 --2- 172.21.15.181:0/3679392933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf8104160 0x7f5bf810fac0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5be8000c00 tx=0x7f5be80047b0 comp rx=0 tx=0).stop 2024-05-03T20:34:58.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.208+0000 7f5bfde24700 1 -- 172.21.15.181:0/3679392933 reap_dead start 2024-05-03T20:34:58.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.208+0000 7f5be3fff700 1 -- 172.21.15.181:0/3679392933 shutdown_connections 2024-05-03T20:34:58.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.208+0000 7f5be3fff700 1 -- 172.21.15.181:0/3679392933 >> 172.21.15.181:0/3679392933 conn(0x7f5bf807a580 msgr2=0x7f5bf8102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:34:58.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.208+0000 7f5be3fff700 1 -- 172.21.15.181:0/3679392933 shutdown_connections 2024-05-03T20:34:58.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:34:58.208+0000 7f5be3fff700 1 -- 172.21.15.181:0/3679392933 wait complete. 2024-05-03T20:34:58.616 INFO:teuthology.orchestra.run.smithi181.stdout:115964117061 2024-05-03T20:34:58.616 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-03T20:34:59.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:59 smithi196 ceph-mon[32858]: pgmap v325: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-05-03T20:34:59.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:59 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:34:59.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:59 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:34:59.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:59 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:34:59.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:59 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:34:59.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:34:59 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:34:59.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:59 smithi204 ceph-mon[33595]: pgmap v325: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-05-03T20:34:59.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:59 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:34:59.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:59 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:34:59.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:59 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:34:59.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:59 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:34:59.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:34:59 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:34:59.399 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:59 smithi181 ceph-mon[29801]: pgmap v325: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-05-03T20:34:59.400 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:59 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:34:59.400 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:59 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:34:59.400 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:59 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:34:59.400 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:59 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:34:59.400 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:34:59 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:34:59.950 INFO:teuthology.orchestra.run.smithi181.stdout:287762808867 2024-05-03T20:34:59.951 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-03T20:35:00.329 INFO:teuthology.orchestra.run.smithi181.stdout:210453397554 2024-05-03T20:35:00.329 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-03T20:35:00.750 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:35:01.315 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:01 smithi181 ceph-mon[29801]: pgmap v326: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:35:01.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.318+0000 7f2fee64e700 1 -- 172.21.15.181:0/3768775205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe8104160 msgr2=0x7f2fe8108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:01.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.318+0000 7f2fee64e700 1 --2- 172.21.15.181:0/3768775205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe8104160 0x7f2fe8108150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0099c0 tx=0x7f2fdc009cd0 comp rx=0 tx=0).stop 2024-05-03T20:35:01.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.318+0000 7f2fee64e700 1 -- 172.21.15.181:0/3768775205 shutdown_connections 2024-05-03T20:35:01.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.318+0000 7f2fee64e700 1 --2- 172.21.15.181:0/3768775205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2fe8108880 0x7f2fe8110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:01.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.318+0000 7f2fee64e700 1 --2- 172.21.15.181:0/3768775205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fe81037b0 0x7f2fe8103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:01.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.318+0000 7f2fee64e700 1 --2- 172.21.15.181:0/3768775205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe8104160 0x7f2fe8108150 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:01.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.318+0000 7f2fee64e700 1 -- 172.21.15.181:0/3768775205 >> 172.21.15.181:0/3768775205 conn(0x7f2fe807a530 msgr2=0x7f2fe807a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:01.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.318+0000 7f2fee64e700 1 -- 172.21.15.181:0/3768775205 shutdown_connections 2024-05-03T20:35:01.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.319+0000 7f2fee64e700 1 -- 172.21.15.181:0/3768775205 wait complete. 2024-05-03T20:35:01.319 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.319+0000 7f2fee64e700 1 Processor -- start 2024-05-03T20:35:01.319 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.319+0000 7f2fee64e700 1 -- start start 2024-05-03T20:35:01.319 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.319+0000 7f2fee64e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe81037b0 0x7f2fe819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:01.319 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.319+0000 7f2fee64e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2fe8104160 0x7f2fe819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:01.319 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.319+0000 7f2fee64e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fe8108880 0x7f2fe8199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:01.319 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.319+0000 7f2fee64e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe81136c0 con 0x7f2fe81037b0 2024-05-03T20:35:01.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.319+0000 7f2fee64e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe8113540 con 0x7f2fe8104160 2024-05-03T20:35:01.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.319+0000 7f2fee64e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe8113840 con 0x7f2fe8108880 2024-05-03T20:35:01.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.320+0000 7f2fe7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe81037b0 0x7f2fe819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:01.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.320+0000 7f2fe7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe81037b0 0x7f2fe819f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50724/0 (socket says 172.21.15.181:50724) 2024-05-03T20:35:01.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.320+0000 7f2fe7fff700 1 -- 172.21.15.181:0/2152046788 learned_addr learned my addr 172.21.15.181:0/2152046788 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:01.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.320+0000 7f2fe77fe700 1 --2- 172.21.15.181:0/2152046788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2fe8104160 0x7f2fe819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:01.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.320+0000 7f2fe7fff700 1 -- 172.21.15.181:0/2152046788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fe8108880 msgr2=0x7f2fe8199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:01.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.320+0000 7f2fe7fff700 1 --2- 172.21.15.181:0/2152046788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fe8108880 0x7f2fe8199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:01.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.320+0000 7f2fe7fff700 1 -- 172.21.15.181:0/2152046788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2fe8104160 msgr2=0x7f2fe819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:01.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.320+0000 7f2fe7fff700 1 --2- 172.21.15.181:0/2152046788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2fe8104160 0x7f2fe819fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:01.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.320+0000 7f2fe7fff700 1 -- 172.21.15.181:0/2152046788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe819a060 con 0x7f2fe81037b0 2024-05-03T20:35:01.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.320+0000 7f2fe7fff700 1 --2- 172.21.15.181:0/2152046788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe81037b0 0x7f2fe819f740 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800b530 tx=0x7f2fd800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:01.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.320+0000 7f2fe57fa700 1 -- 172.21.15.181:0/2152046788 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8015020 con 0x7f2fe81037b0 2024-05-03T20:35:01.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.320+0000 7f2fe57fa700 1 -- 172.21.15.181:0/2152046788 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fd8011ce0 con 0x7f2fe81037b0 2024-05-03T20:35:01.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.320+0000 7f2fee64e700 1 -- 172.21.15.181:0/2152046788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe819a350 con 0x7f2fe81037b0 2024-05-03T20:35:01.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.320+0000 7f2fe57fa700 1 -- 172.21.15.181:0/2152046788 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8010d20 con 0x7f2fe81037b0 2024-05-03T20:35:01.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.320+0000 7f2fee64e700 1 -- 172.21.15.181:0/2152046788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe81a6610 con 0x7f2fe81037b0 2024-05-03T20:35:01.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.321+0000 7f2fee64e700 1 -- 172.21.15.181:0/2152046788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2fe8063bf0 con 0x7f2fe81037b0 2024-05-03T20:35:01.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.322+0000 7f2fe57fa700 1 -- 172.21.15.181:0/2152046788 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f2fd8018020 con 0x7f2fe81037b0 2024-05-03T20:35:01.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.323+0000 7f2fe57fa700 1 --2- 172.21.15.181:0/2152046788 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f2fd00b2910 0x7f2fd00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:01.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.323+0000 7f2fe77fe700 1 --2- 172.21.15.181:0/2152046788 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f2fd00b2910 0x7f2fd00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:01.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.323+0000 7f2fe57fa700 1 -- 172.21.15.181:0/2152046788 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f2fd80e2280 con 0x7f2fe81037b0 2024-05-03T20:35:01.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.323+0000 7f2fe57fa700 1 --2- 172.21.15.181:0/2152046788 >> [v2:172.21.15.181:6818/1561969338,v1:172.21.15.181:6819/1561969338] conn(0x7f2fd00bc450 0x7f2fd00be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:01.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.323+0000 7f2fe57fa700 1 -- 172.21.15.181:0/2152046788 --> [v2:172.21.15.181:6818/1561969338,v1:172.21.15.181:6819/1561969338] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2fd801be50 con 0x7f2fd00bc450 2024-05-03T20:35:01.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.323+0000 7f2fe57fa700 1 -- 172.21.15.181:0/2152046788 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f2fd80e26d0 con 0x7f2fe81037b0 2024-05-03T20:35:01.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.323+0000 7f2fecbeb700 1 --2- 172.21.15.181:0/2152046788 >> [v2:172.21.15.181:6818/1561969338,v1:172.21.15.181:6819/1561969338] conn(0x7f2fd00bc450 0x7f2fd00be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:01.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.324+0000 7f2fe77fe700 1 --2- 172.21.15.181:0/2152046788 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f2fd00b2910 0x7f2fd00b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc003800 tx=0x7f2fdc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:01.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.324+0000 7f2fecbeb700 1 --2- 172.21.15.181:0/2152046788 >> [v2:172.21.15.181:6818/1561969338,v1:172.21.15.181:6819/1561969338] conn(0x7f2fd00bc450 0x7f2fd00be8f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:01.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.324+0000 7f2fe57fa700 1 -- 172.21.15.181:0/2152046788 <== osd.6 v2:172.21.15.181:6818/1561969338 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2fd801be50 con 0x7f2fd00bc450 2024-05-03T20:35:01.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.343+0000 7f2fee64e700 1 -- 172.21.15.181:0/2152046788 --> [v2:172.21.15.181:6818/1561969338,v1:172.21.15.181:6819/1561969338] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2fe8064070 con 0x7f2fd00bc450 2024-05-03T20:35:01.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.344+0000 7f2fe57fa700 1 -- 172.21.15.181:0/2152046788 <== osd.6 v2:172.21.15.181:6818/1561969338 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2fe8064070 con 0x7f2fd00bc450 2024-05-03T20:35:01.343 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.344+0000 7f2fee64e700 1 -- 172.21.15.181:0/2152046788 >> [v2:172.21.15.181:6818/1561969338,v1:172.21.15.181:6819/1561969338] conn(0x7f2fd00bc450 msgr2=0x7f2fd00be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:01.343 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.344+0000 7f2fee64e700 1 --2- 172.21.15.181:0/2152046788 >> [v2:172.21.15.181:6818/1561969338,v1:172.21.15.181:6819/1561969338] conn(0x7f2fd00bc450 0x7f2fd00be8f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:01.343 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.344+0000 7f2fee64e700 1 -- 172.21.15.181:0/2152046788 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f2fd00b2910 msgr2=0x7f2fd00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:01.343 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.344+0000 7f2fee64e700 1 --2- 172.21.15.181:0/2152046788 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f2fd00b2910 0x7f2fd00b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc003800 tx=0x7f2fdc000bc0 comp rx=0 tx=0).stop 2024-05-03T20:35:01.343 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.344+0000 7f2fee64e700 1 -- 172.21.15.181:0/2152046788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe81037b0 msgr2=0x7f2fe819f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:01.343 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.344+0000 7f2fee64e700 1 --2- 172.21.15.181:0/2152046788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe81037b0 0x7f2fe819f740 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800b530 tx=0x7f2fd800b8f0 comp rx=0 tx=0).stop 2024-05-03T20:35:01.343 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.344+0000 7f2fecbeb700 1 -- 172.21.15.181:0/2152046788 reap_dead start 2024-05-03T20:35:01.343 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.344+0000 7f2fee64e700 1 -- 172.21.15.181:0/2152046788 shutdown_connections 2024-05-03T20:35:01.343 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.344+0000 7f2fee64e700 1 -- 172.21.15.181:0/2152046788 >> 172.21.15.181:0/2152046788 conn(0x7f2fe807a530 msgr2=0x7f2fe8102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:01.343 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.345+0000 7f2fee64e700 1 -- 172.21.15.181:0/2152046788 shutdown_connections 2024-05-03T20:35:01.343 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:01.345+0000 7f2fee64e700 1 -- 172.21.15.181:0/2152046788 wait complete. 2024-05-03T20:35:01.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:01 smithi196 ceph-mon[32858]: pgmap v326: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:35:01.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:01 smithi204 ceph-mon[33595]: pgmap v326: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:35:01.821 INFO:teuthology.orchestra.run.smithi181.stdout:55834574929 2024-05-03T20:35:01.821 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-03T20:35:02.283 INFO:teuthology.orchestra.run.smithi181.stdout:313532612637 2024-05-03T20:35:02.283 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-03T20:35:03.262 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:03 smithi181 ceph-mon[29801]: pgmap v327: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:35:03.294 INFO:teuthology.orchestra.run.smithi181.stdout:188978561080 2024-05-03T20:35:03.295 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-03T20:35:03.301 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:35:03.301 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:35:03.302 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:35:03.303 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:35:03.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:03 smithi196 ceph-mon[32858]: pgmap v327: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:35:03.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:03 smithi204 ceph-mon[33595]: pgmap v327: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:35:05.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.150+0000 7fd0522c5700 1 -- 172.21.15.181:0/3746046167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd04c109520 msgr2=0x7fd04c109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:05.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.150+0000 7fd0522c5700 1 --2- 172.21.15.181:0/3746046167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd04c109520 0x7fd04c109900 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd03c009990 tx=0x7fd03c009ca0 comp rx=0 tx=0).stop 2024-05-03T20:35:05.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.150+0000 7fd0522c5700 1 -- 172.21.15.181:0/3746046167 shutdown_connections 2024-05-03T20:35:05.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.150+0000 7fd0522c5700 1 --2- 172.21.15.181:0/3746046167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd04c102450 0x7fd04c110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:05.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.150+0000 7fd0522c5700 1 --2- 172.21.15.181:0/3746046167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd04c109520 0x7fd04c109900 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:05.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.150+0000 7fd0522c5700 1 --2- 172.21.15.181:0/3746046167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd04c101a90 0x7fd04c101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:05.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.150+0000 7fd0522c5700 1 -- 172.21.15.181:0/3746046167 >> 172.21.15.181:0/3746046167 conn(0x7fd04c078a10 msgr2=0x7fd04c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:05.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.150+0000 7fd0522c5700 1 -- 172.21.15.181:0/3746046167 shutdown_connections 2024-05-03T20:35:05.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.150+0000 7fd0522c5700 1 -- 172.21.15.181:0/3746046167 wait complete. 2024-05-03T20:35:05.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.151+0000 7fd0522c5700 1 Processor -- start 2024-05-03T20:35:05.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.151+0000 7fd0522c5700 1 -- start start 2024-05-03T20:35:05.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.151+0000 7fd0522c5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd04c101a90 0x7fd04c102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:05.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.151+0000 7fd0522c5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd04c102450 0x7fd04c1031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:05.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.151+0000 7fd0522c5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd04c109520 0x7fd04c1036f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:05.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.151+0000 7fd0522c5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c113700 con 0x7fd04c101a90 2024-05-03T20:35:05.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.151+0000 7fd0522c5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c113580 con 0x7fd04c109520 2024-05-03T20:35:05.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.151+0000 7fd0522c5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c113880 con 0x7fd04c102450 2024-05-03T20:35:05.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.151+0000 7fd04bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd04c101a90 0x7fd04c102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:05.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.151+0000 7fd04bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd04c101a90 0x7fd04c102c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50736/0 (socket says 172.21.15.181:50736) 2024-05-03T20:35:05.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.151+0000 7fd04bfff700 1 -- 172.21.15.181:0/1421302827 learned_addr learned my addr 172.21.15.181:0/1421302827 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:05.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.151+0000 7fd050862700 1 --2- 172.21.15.181:0/1421302827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd04c109520 0x7fd04c1036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:05.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.151+0000 7fd04b7fe700 1 --2- 172.21.15.181:0/1421302827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd04c102450 0x7fd04c1031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:05.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.151+0000 7fd04bfff700 1 -- 172.21.15.181:0/1421302827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd04c102450 msgr2=0x7fd04c1031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:05.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.151+0000 7fd04bfff700 1 --2- 172.21.15.181:0/1421302827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd04c102450 0x7fd04c1031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:05.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.151+0000 7fd04bfff700 1 -- 172.21.15.181:0/1421302827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd04c109520 msgr2=0x7fd04c1036f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:05.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.151+0000 7fd04bfff700 1 --2- 172.21.15.181:0/1421302827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd04c109520 0x7fd04c1036f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:05.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.151+0000 7fd04bfff700 1 -- 172.21.15.181:0/1421302827 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04c1a8120 con 0x7fd04c101a90 2024-05-03T20:35:05.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.151+0000 7fd04b7fe700 1 --2- 172.21.15.181:0/1421302827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd04c102450 0x7fd04c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:35:05.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.152+0000 7fd04bfff700 1 --2- 172.21.15.181:0/1421302827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd04c101a90 0x7fd04c102c70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd03c005910 tx=0x7fd03c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:05.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.152+0000 7fd0497fa700 1 -- 172.21.15.181:0/1421302827 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c027070 con 0x7fd04c101a90 2024-05-03T20:35:05.153 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.152+0000 7fd0497fa700 1 -- 172.21.15.181:0/1421302827 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd03c003f20 con 0x7fd04c101a90 2024-05-03T20:35:05.153 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.152+0000 7fd0522c5700 1 -- 172.21.15.181:0/1421302827 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd04c1a83b0 con 0x7fd04c101a90 2024-05-03T20:35:05.153 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.152+0000 7fd0497fa700 1 -- 172.21.15.181:0/1421302827 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c022c70 con 0x7fd04c101a90 2024-05-03T20:35:05.153 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.152+0000 7fd0522c5700 1 -- 172.21.15.181:0/1421302827 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd04c1a8800 con 0x7fd04c101a90 2024-05-03T20:35:05.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.153+0000 7fd0522c5700 1 -- 172.21.15.181:0/1421302827 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd04c063bb0 con 0x7fd04c101a90 2024-05-03T20:35:05.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.154+0000 7fd0497fa700 1 -- 172.21.15.181:0/1421302827 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fd03c02a070 con 0x7fd04c101a90 2024-05-03T20:35:05.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.155+0000 7fd0497fa700 1 --2- 172.21.15.181:0/1421302827 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd0340b2910 0x7fd0340b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:05.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.155+0000 7fd04b7fe700 1 --2- 172.21.15.181:0/1421302827 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd0340b2910 0x7fd0340b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:05.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.155+0000 7fd0497fa700 1 -- 172.21.15.181:0/1421302827 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7fd03c0e3c90 con 0x7fd04c101a90 2024-05-03T20:35:05.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.155+0000 7fd0497fa700 1 --2- 172.21.15.181:0/1421302827 >> [v2:172.21.15.181:6802/1683973278,v1:172.21.15.181:6803/1683973278] conn(0x7fd0340bc450 0x7fd0340be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:05.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.155+0000 7fd0497fa700 1 -- 172.21.15.181:0/1421302827 --> [v2:172.21.15.181:6802/1683973278,v1:172.21.15.181:6803/1683973278] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd03c0056d0 con 0x7fd0340bc450 2024-05-03T20:35:05.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.155+0000 7fd0497fa700 1 -- 172.21.15.181:0/1421302827 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fd03c0e40e0 con 0x7fd04c101a90 2024-05-03T20:35:05.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.155+0000 7fd050862700 1 --2- 172.21.15.181:0/1421302827 >> [v2:172.21.15.181:6802/1683973278,v1:172.21.15.181:6803/1683973278] conn(0x7fd0340bc450 0x7fd0340be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:05.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.155+0000 7fd04b7fe700 1 --2- 172.21.15.181:0/1421302827 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd0340b2910 0x7fd0340b4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd04c0ffb10 tx=0x7fd040008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:05.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.156+0000 7fd050862700 1 --2- 172.21.15.181:0/1421302827 >> [v2:172.21.15.181:6802/1683973278,v1:172.21.15.181:6803/1683973278] conn(0x7fd0340bc450 0x7fd0340be8f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:05.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.156+0000 7fd0497fa700 1 -- 172.21.15.181:0/1421302827 <== osd.0 v2:172.21.15.181:6802/1683973278 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd03c0056d0 con 0x7fd0340bc450 2024-05-03T20:35:05.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.173+0000 7fd0522c5700 1 -- 172.21.15.181:0/1421302827 --> [v2:172.21.15.181:6802/1683973278,v1:172.21.15.181:6803/1683973278] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd04c06a190 con 0x7fd0340bc450 2024-05-03T20:35:05.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.173+0000 7fd0497fa700 1 -- 172.21.15.181:0/1421302827 <== osd.0 v2:172.21.15.181:6802/1683973278 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd04c06a190 con 0x7fd0340bc450 2024-05-03T20:35:05.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.174+0000 7fd0522c5700 1 -- 172.21.15.181:0/1421302827 >> [v2:172.21.15.181:6802/1683973278,v1:172.21.15.181:6803/1683973278] conn(0x7fd0340bc450 msgr2=0x7fd0340be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:05.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.174+0000 7fd0522c5700 1 --2- 172.21.15.181:0/1421302827 >> [v2:172.21.15.181:6802/1683973278,v1:172.21.15.181:6803/1683973278] conn(0x7fd0340bc450 0x7fd0340be8f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:05.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.174+0000 7fd0522c5700 1 -- 172.21.15.181:0/1421302827 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd0340b2910 msgr2=0x7fd0340b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:05.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.174+0000 7fd0522c5700 1 --2- 172.21.15.181:0/1421302827 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd0340b2910 0x7fd0340b4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd04c0ffb10 tx=0x7fd040008040 comp rx=0 tx=0).stop 2024-05-03T20:35:05.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.174+0000 7fd0522c5700 1 -- 172.21.15.181:0/1421302827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd04c101a90 msgr2=0x7fd04c102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:05.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.174+0000 7fd0522c5700 1 --2- 172.21.15.181:0/1421302827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd04c101a90 0x7fd04c102c70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd03c005910 tx=0x7fd03c0047b0 comp rx=0 tx=0).stop 2024-05-03T20:35:05.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.174+0000 7fd050862700 1 -- 172.21.15.181:0/1421302827 reap_dead start 2024-05-03T20:35:05.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.174+0000 7fd0522c5700 1 -- 172.21.15.181:0/1421302827 shutdown_connections 2024-05-03T20:35:05.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.174+0000 7fd0522c5700 1 -- 172.21.15.181:0/1421302827 >> 172.21.15.181:0/1421302827 conn(0x7fd04c078a10 msgr2=0x7fd04c1002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:05.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.174+0000 7fd0522c5700 1 -- 172.21.15.181:0/1421302827 shutdown_connections 2024-05-03T20:35:05.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:05.174+0000 7fd0522c5700 1 -- 172.21.15.181:0/1421302827 wait complete. 2024-05-03T20:35:05.352 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:05 smithi181 ceph-mon[29801]: pgmap v328: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:35:05.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:05 smithi196 ceph-mon[32858]: pgmap v328: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:35:05.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:05 smithi204 ceph-mon[33595]: pgmap v328: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:35:06.726 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:35:06.726 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:35:07.184 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:07 smithi181 ceph-mon[29801]: pgmap v329: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-03T20:35:07.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:07 smithi196 ceph-mon[32858]: pgmap v329: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-03T20:35:07.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:07 smithi204 ceph-mon[33595]: pgmap v329: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-03T20:35:07.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.469+0000 7fea1ce69700 1 -- 172.21.15.181:0/3922529568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea181076e0 msgr2=0x7fea1810f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:07.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.469+0000 7fea1ce69700 1 --2- 172.21.15.181:0/3922529568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea181076e0 0x7fea1810f2b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fea1000b0d0 tx=0x7fea1000b3e0 comp rx=0 tx=0).stop 2024-05-03T20:35:07.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.469+0000 7fea1ce69700 1 -- 172.21.15.181:0/3922529568 shutdown_connections 2024-05-03T20:35:07.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.469+0000 7fea1ce69700 1 --2- 172.21.15.181:0/3922529568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea181076e0 0x7fea1810f2b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:07.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.469+0000 7fea1ce69700 1 --2- 172.21.15.181:0/3922529568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea18102690 0x7fea18102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:07.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.469+0000 7fea1ce69700 1 --2- 172.21.15.181:0/3922529568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea18103040 0x7fea18107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:07.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.469+0000 7fea1ce69700 1 -- 172.21.15.181:0/3922529568 >> 172.21.15.181:0/3922529568 conn(0x7fea180783f0 msgr2=0x7fea18078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:07.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.469+0000 7fea1ce69700 1 -- 172.21.15.181:0/3922529568 shutdown_connections 2024-05-03T20:35:07.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.470+0000 7fea1ce69700 1 -- 172.21.15.181:0/3922529568 wait complete. 2024-05-03T20:35:07.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.470+0000 7fea1ce69700 1 Processor -- start 2024-05-03T20:35:07.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.470+0000 7fea1ce69700 1 -- start start 2024-05-03T20:35:07.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.471+0000 7fea1ce69700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea18102690 0x7fea18198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:07.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.471+0000 7fea1ce69700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea18103040 0x7fea18198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:07.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.471+0000 7fea1ce69700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea181076e0 0x7fea18199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:07.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.471+0000 7fea1ce69700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fea181125f0 con 0x7fea18102690 2024-05-03T20:35:07.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.471+0000 7fea1ce69700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fea18112470 con 0x7fea18103040 2024-05-03T20:35:07.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.471+0000 7fea1ce69700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fea18112770 con 0x7fea181076e0 2024-05-03T20:35:07.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.471+0000 7fea1659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea18102690 0x7fea18198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:07.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.471+0000 7fea1659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea18102690 0x7fea18198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33882/0 (socket says 172.21.15.181:33882) 2024-05-03T20:35:07.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.471+0000 7fea1659c700 1 -- 172.21.15.181:0/1330153553 learned_addr learned my addr 172.21.15.181:0/1330153553 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:07.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.471+0000 7fea15d9b700 1 --2- 172.21.15.181:0/1330153553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea18103040 0x7fea18198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:07.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.471+0000 7fea16d9d700 1 --2- 172.21.15.181:0/1330153553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea181076e0 0x7fea18199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:07.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.471+0000 7fea1659c700 1 -- 172.21.15.181:0/1330153553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea181076e0 msgr2=0x7fea18199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:07.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.471+0000 7fea1659c700 1 --2- 172.21.15.181:0/1330153553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea181076e0 0x7fea18199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:07.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.471+0000 7fea1659c700 1 -- 172.21.15.181:0/1330153553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea18103040 msgr2=0x7fea18198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:07.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.471+0000 7fea1659c700 1 --2- 172.21.15.181:0/1330153553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea18103040 0x7fea18198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:07.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.471+0000 7fea1659c700 1 -- 172.21.15.181:0/1330153553 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea181a8010 con 0x7fea18102690 2024-05-03T20:35:07.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.471+0000 7fea1659c700 1 --2- 172.21.15.181:0/1330153553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea18102690 0x7fea18198800 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fea0c00c920 tx=0x7fea0c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:07.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.471+0000 7fea077fe700 1 -- 172.21.15.181:0/1330153553 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c007b00 con 0x7fea18102690 2024-05-03T20:35:07.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.471+0000 7fea1ce69700 1 -- 172.21.15.181:0/1330153553 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea181a8300 con 0x7fea18102690 2024-05-03T20:35:07.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.471+0000 7fea077fe700 1 -- 172.21.15.181:0/1330153553 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea0c004ce0 con 0x7fea18102690 2024-05-03T20:35:07.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.471+0000 7fea077fe700 1 -- 172.21.15.181:0/1330153553 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c005500 con 0x7fea18102690 2024-05-03T20:35:07.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.471+0000 7fea1ce69700 1 -- 172.21.15.181:0/1330153553 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea181a8890 con 0x7fea18102690 2024-05-03T20:35:07.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.472+0000 7fea1ce69700 1 -- 172.21.15.181:0/1330153553 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea1806a190 con 0x7fea18102690 2024-05-03T20:35:07.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.474+0000 7fea077fe700 1 -- 172.21.15.181:0/1330153553 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fea0c004750 con 0x7fea18102690 2024-05-03T20:35:07.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.474+0000 7fea077fe700 1 --2- 172.21.15.181:0/1330153553 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fea000b2910 0x7fea000b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:07.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.474+0000 7fea15d9b700 1 --2- 172.21.15.181:0/1330153553 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fea000b2910 0x7fea000b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:07.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.474+0000 7fea077fe700 1 -- 172.21.15.181:0/1330153553 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7fea0c0e2330 con 0x7fea18102690 2024-05-03T20:35:07.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.475+0000 7fea15d9b700 1 --2- 172.21.15.181:0/1330153553 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fea000b2910 0x7fea000b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fea1819a960 tx=0x7fea08009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:07.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.478+0000 7fea077fe700 1 -- 172.21.15.181:0/1330153553 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fea0c014030 con 0x7fea18102690 2024-05-03T20:35:07.697 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.698+0000 7fea1ce69700 1 -- 172.21.15.181:0/1330153553 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fea1806c1e0 con 0x7fea18102690 2024-05-03T20:35:07.697 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.698+0000 7fea077fe700 1 -- 172.21.15.181:0/1330153553 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fea1806c1e0 con 0x7fea18102690 2024-05-03T20:35:07.697 INFO:teuthology.orchestra.run.smithi181.stdout:240518168624 2024-05-03T20:35:07.699 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.701+0000 7fea1ce69700 1 -- 172.21.15.181:0/1330153553 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fea000b2910 msgr2=0x7fea000b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:07.699 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.701+0000 7fea1ce69700 1 --2- 172.21.15.181:0/1330153553 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fea000b2910 0x7fea000b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fea1819a960 tx=0x7fea08009190 comp rx=0 tx=0).stop 2024-05-03T20:35:07.699 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.701+0000 7fea1ce69700 1 -- 172.21.15.181:0/1330153553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea18102690 msgr2=0x7fea18198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:07.700 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.701+0000 7fea1ce69700 1 --2- 172.21.15.181:0/1330153553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea18102690 0x7fea18198800 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fea0c00c920 tx=0x7fea0c00cc30 comp rx=0 tx=0).stop 2024-05-03T20:35:07.700 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.701+0000 7fea1ce69700 1 -- 172.21.15.181:0/1330153553 shutdown_connections 2024-05-03T20:35:07.700 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.701+0000 7fea1ce69700 1 --2- 172.21.15.181:0/1330153553 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fea000b2910 0x7fea000b4dd0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:07.700 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.701+0000 7fea1ce69700 1 --2- 172.21.15.181:0/1330153553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea181076e0 0x7fea18199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:07.700 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.701+0000 7fea1ce69700 1 --2- 172.21.15.181:0/1330153553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea18102690 0x7fea18198800 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:07.700 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.701+0000 7fea1ce69700 1 --2- 172.21.15.181:0/1330153553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea18103040 0x7fea18198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:07.700 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.701+0000 7fea1ce69700 1 -- 172.21.15.181:0/1330153553 >> 172.21.15.181:0/1330153553 conn(0x7fea180783f0 msgr2=0x7fea18100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:07.700 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.701+0000 7fea1ce69700 1 -- 172.21.15.181:0/1330153553 shutdown_connections 2024-05-03T20:35:07.700 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:07.701+0000 7fea1ce69700 1 -- 172.21.15.181:0/1330153553 wait complete. 2024-05-03T20:35:08.340 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:08 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/1330153553' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-03T20:35:08.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:08 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/1330153553' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-03T20:35:08.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:08 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/1330153553' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-03T20:35:09.345 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:09 smithi181 ceph-mon[29801]: pgmap v330: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-03T20:35:09.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:09 smithi196 ceph-mon[32858]: pgmap v330: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-03T20:35:09.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:09 smithi204 ceph-mon[33595]: pgmap v330: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-03T20:35:09.976 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168620 got 240518168624 for osd.8 2024-05-03T20:35:09.977 DEBUG:teuthology.parallel:result is None 2024-05-03T20:35:10.991 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.992+0000 7fd045f4d700 1 -- 172.21.15.181:0/2448997519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd040075ff0 msgr2=0x7fd04010f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:10.991 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.992+0000 7fd045f4d700 1 --2- 172.21.15.181:0/2448997519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd040075ff0 0x7fd04010f960 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd030009990 tx=0x7fd030009ca0 comp rx=0 tx=0).stop 2024-05-03T20:35:10.991 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.992+0000 7fd045f4d700 1 -- 172.21.15.181:0/2448997519 shutdown_connections 2024-05-03T20:35:10.992 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.992+0000 7fd045f4d700 1 --2- 172.21.15.181:0/2448997519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd04010fea0 0x7fd040112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:10.992 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.992+0000 7fd045f4d700 1 --2- 172.21.15.181:0/2448997519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0400756d0 0x7fd040075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:10.992 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.992+0000 7fd045f4d700 1 --2- 172.21.15.181:0/2448997519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd040075ff0 0x7fd04010f960 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:10.992 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.992+0000 7fd045f4d700 1 -- 172.21.15.181:0/2448997519 >> 172.21.15.181:0/2448997519 conn(0x7fd04007a600 msgr2=0x7fd04007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:10.992 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.992+0000 7fd045f4d700 1 -- 172.21.15.181:0/2448997519 shutdown_connections 2024-05-03T20:35:10.992 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.992+0000 7fd045f4d700 1 -- 172.21.15.181:0/2448997519 wait complete. 2024-05-03T20:35:10.992 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.993+0000 7fd045f4d700 1 Processor -- start 2024-05-03T20:35:10.992 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.993+0000 7fd045f4d700 1 -- start start 2024-05-03T20:35:10.994 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.993+0000 7fd045f4d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0400756d0 0x7fd04019be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:10.994 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.993+0000 7fd045f4d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd040075ff0 0x7fd04019c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:10.994 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.993+0000 7fd045f4d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd04010fea0 0x7fd0401a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:10.994 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.993+0000 7fd045f4d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd0401158c0 con 0x7fd040075ff0 2024-05-03T20:35:10.994 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.993+0000 7fd045f4d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd040115740 con 0x7fd04010fea0 2024-05-03T20:35:10.994 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.993+0000 7fd045f4d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd040115a40 con 0x7fd0400756d0 2024-05-03T20:35:10.994 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.994+0000 7fd03effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd040075ff0 0x7fd04019c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:10.994 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.994+0000 7fd03effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd040075ff0 0x7fd04019c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33900/0 (socket says 172.21.15.181:33900) 2024-05-03T20:35:10.994 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.994+0000 7fd03effd700 1 -- 172.21.15.181:0/3352977741 learned_addr learned my addr 172.21.15.181:0/3352977741 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:10.994 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.994+0000 7fd03f7fe700 1 --2- 172.21.15.181:0/3352977741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0400756d0 0x7fd04019be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:10.994 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.994+0000 7fd03ffff700 1 --2- 172.21.15.181:0/3352977741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd04010fea0 0x7fd0401a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:10.994 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.994+0000 7fd03effd700 1 -- 172.21.15.181:0/3352977741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0400756d0 msgr2=0x7fd04019be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:10.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.994+0000 7fd03effd700 1 --2- 172.21.15.181:0/3352977741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0400756d0 0x7fd04019be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:10.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.994+0000 7fd03effd700 1 -- 172.21.15.181:0/3352977741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd04010fea0 msgr2=0x7fd0401a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:10.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.994+0000 7fd03effd700 1 --2- 172.21.15.181:0/3352977741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd04010fea0 0x7fd0401a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:10.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.994+0000 7fd03effd700 1 -- 172.21.15.181:0/3352977741 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0401a0e10 con 0x7fd040075ff0 2024-05-03T20:35:10.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.994+0000 7fd03ffff700 1 --2- 172.21.15.181:0/3352977741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd04010fea0 0x7fd0401a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:35:10.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.994+0000 7fd03effd700 1 --2- 172.21.15.181:0/3352977741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd040075ff0 0x7fd04019c390 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd03000b5c0 tx=0x7fd030004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:10.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.994+0000 7fd03cff9700 1 -- 172.21.15.181:0/3352977741 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd030026070 con 0x7fd040075ff0 2024-05-03T20:35:10.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.994+0000 7fd03cff9700 1 -- 172.21.15.181:0/3352977741 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0300040f0 con 0x7fd040075ff0 2024-05-03T20:35:10.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.994+0000 7fd045f4d700 1 -- 172.21.15.181:0/3352977741 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0401a10a0 con 0x7fd040075ff0 2024-05-03T20:35:10.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.994+0000 7fd03cff9700 1 -- 172.21.15.181:0/3352977741 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0300052b0 con 0x7fd040075ff0 2024-05-03T20:35:10.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.994+0000 7fd045f4d700 1 -- 172.21.15.181:0/3352977741 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0401a8980 con 0x7fd040075ff0 2024-05-03T20:35:10.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.995+0000 7fd045f4d700 1 -- 172.21.15.181:0/3352977741 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd040076dc0 con 0x7fd040075ff0 2024-05-03T20:35:10.996 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.996+0000 7fd03cff9700 1 -- 172.21.15.181:0/3352977741 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fd030029070 con 0x7fd040075ff0 2024-05-03T20:35:10.996 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.997+0000 7fd03cff9700 1 --2- 172.21.15.181:0/3352977741 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd0280b2910 0x7fd0280b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:10.996 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.997+0000 7fd03f7fe700 1 --2- 172.21.15.181:0/3352977741 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd0280b2910 0x7fd0280b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:10.996 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.997+0000 7fd03cff9700 1 -- 172.21.15.181:0/3352977741 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7fd0300e2fd0 con 0x7fd040075ff0 2024-05-03T20:35:10.996 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:10.997+0000 7fd03f7fe700 1 --2- 172.21.15.181:0/3352977741 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd0280b2910 0x7fd0280b4dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd034005950 tx=0x7fd03400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:11.000 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.001+0000 7fd03cff9700 1 -- 172.21.15.181:0/3352977741 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd030013070 con 0x7fd040075ff0 2024-05-03T20:35:11.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.225+0000 7fd045f4d700 1 -- 172.21.15.181:0/3352977741 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fd04002e510 con 0x7fd040075ff0 2024-05-03T20:35:11.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.226+0000 7fd03cff9700 1 -- 172.21.15.181:0/3352977741 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd0300aac20 con 0x7fd040075ff0 2024-05-03T20:35:11.225 INFO:teuthology.orchestra.run.smithi181.stdout:266287972396 2024-05-03T20:35:11.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.228+0000 7fd045f4d700 1 -- 172.21.15.181:0/3352977741 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd0280b2910 msgr2=0x7fd0280b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:11.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.228+0000 7fd045f4d700 1 --2- 172.21.15.181:0/3352977741 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd0280b2910 0x7fd0280b4dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd034005950 tx=0x7fd03400a300 comp rx=0 tx=0).stop 2024-05-03T20:35:11.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.228+0000 7fd045f4d700 1 -- 172.21.15.181:0/3352977741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd040075ff0 msgr2=0x7fd04019c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:11.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.228+0000 7fd045f4d700 1 --2- 172.21.15.181:0/3352977741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd040075ff0 0x7fd04019c390 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd03000b5c0 tx=0x7fd030004580 comp rx=0 tx=0).stop 2024-05-03T20:35:11.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.228+0000 7fd045f4d700 1 -- 172.21.15.181:0/3352977741 shutdown_connections 2024-05-03T20:35:11.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.228+0000 7fd045f4d700 1 --2- 172.21.15.181:0/3352977741 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd0280b2910 0x7fd0280b4dd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:11.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.228+0000 7fd045f4d700 1 --2- 172.21.15.181:0/3352977741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd04010fea0 0x7fd0401a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:11.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.228+0000 7fd045f4d700 1 --2- 172.21.15.181:0/3352977741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0400756d0 0x7fd04019be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:11.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.228+0000 7fd045f4d700 1 --2- 172.21.15.181:0/3352977741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd040075ff0 0x7fd04019c390 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:11.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.228+0000 7fd045f4d700 1 -- 172.21.15.181:0/3352977741 >> 172.21.15.181:0/3352977741 conn(0x7fd04007a600 msgr2=0x7fd040100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:11.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.228+0000 7fd045f4d700 1 -- 172.21.15.181:0/3352977741 shutdown_connections 2024-05-03T20:35:11.229 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.228+0000 7fd045f4d700 1 -- 172.21.15.181:0/3352977741 wait complete. 2024-05-03T20:35:11.245 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:11 smithi181 ceph-mon[29801]: pgmap v331: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-03T20:35:11.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:11 smithi196 ceph-mon[32858]: pgmap v331: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-03T20:35:11.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:11 smithi204 ceph-mon[33595]: pgmap v331: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-03T20:35:11.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.403+0000 7f47aa843700 1 -- 172.21.15.181:0/1159478350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47a4104130 msgr2=0x7f47a4108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:11.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.403+0000 7f47aa843700 1 --2- 172.21.15.181:0/1159478350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47a4104130 0x7f47a4108140 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f47980099c0 tx=0x7f4798009cd0 comp rx=0 tx=0).stop 2024-05-03T20:35:11.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.403+0000 7f47aa843700 1 -- 172.21.15.181:0/1159478350 shutdown_connections 2024-05-03T20:35:11.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.403+0000 7f47aa843700 1 --2- 172.21.15.181:0/1159478350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47a4108870 0x7f47a41103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:11.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.403+0000 7f47aa843700 1 --2- 172.21.15.181:0/1159478350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47a4103780 0x7f47a4103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:11.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.403+0000 7f47aa843700 1 --2- 172.21.15.181:0/1159478350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47a4104130 0x7f47a4108140 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:11.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.403+0000 7f47aa843700 1 -- 172.21.15.181:0/1159478350 >> 172.21.15.181:0/1159478350 conn(0x7f47a407a610 msgr2=0x7f47a407aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:11.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.403+0000 7f47aa843700 1 -- 172.21.15.181:0/1159478350 shutdown_connections 2024-05-03T20:35:11.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.403+0000 7f47aa843700 1 -- 172.21.15.181:0/1159478350 wait complete. 2024-05-03T20:35:11.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.404+0000 7f47aa843700 1 Processor -- start 2024-05-03T20:35:11.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.404+0000 7f47aa843700 1 -- start start 2024-05-03T20:35:11.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.404+0000 7f47aa843700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47a4103780 0x7f47a419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:11.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.404+0000 7f47aa843700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47a4104130 0x7f47a419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:11.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.404+0000 7f47aa843700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47a4108870 0x7f47a41998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:11.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.404+0000 7f47aa843700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f47a41136b0 con 0x7f47a4104130 2024-05-03T20:35:11.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.404+0000 7f47aa843700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f47a4113530 con 0x7f47a4103780 2024-05-03T20:35:11.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.404+0000 7f47aa843700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f47a4113830 con 0x7f47a4108870 2024-05-03T20:35:11.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.404+0000 7f47a37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47a4104130 0x7f47a419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:11.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.404+0000 7f47a37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47a4104130 0x7f47a419fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33918/0 (socket says 172.21.15.181:33918) 2024-05-03T20:35:11.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.404+0000 7f47a37fe700 1 -- 172.21.15.181:0/3690890584 learned_addr learned my addr 172.21.15.181:0/3690890584 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:11.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.404+0000 7f47a8de0700 1 --2- 172.21.15.181:0/3690890584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47a4108870 0x7f47a41998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:11.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.404+0000 7f47a3fff700 1 --2- 172.21.15.181:0/3690890584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47a4103780 0x7f47a419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:11.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.404+0000 7f47a37fe700 1 -- 172.21.15.181:0/3690890584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47a4108870 msgr2=0x7f47a41998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:11.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.404+0000 7f47a37fe700 1 --2- 172.21.15.181:0/3690890584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47a4108870 0x7f47a41998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:11.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.404+0000 7f47a37fe700 1 -- 172.21.15.181:0/3690890584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47a4103780 msgr2=0x7f47a419f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:11.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.404+0000 7f47a37fe700 1 --2- 172.21.15.181:0/3690890584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47a4103780 0x7f47a419f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:11.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.404+0000 7f47a37fe700 1 -- 172.21.15.181:0/3690890584 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a419a050 con 0x7f47a4104130 2024-05-03T20:35:11.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.405+0000 7f47a3fff700 1 --2- 172.21.15.181:0/3690890584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47a4103780 0x7f47a419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:35:11.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.405+0000 7f47a37fe700 1 --2- 172.21.15.181:0/3690890584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47a4104130 0x7f47a419fc70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f47980098e0 tx=0x7f4798003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:11.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.405+0000 7f47a17fa700 1 -- 172.21.15.181:0/3690890584 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798027070 con 0x7f47a4104130 2024-05-03T20:35:11.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.405+0000 7f47aa843700 1 -- 172.21.15.181:0/3690890584 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a419a2e0 con 0x7f47a4104130 2024-05-03T20:35:11.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.405+0000 7f47a17fa700 1 -- 172.21.15.181:0/3690890584 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47980040a0 con 0x7f47a4104130 2024-05-03T20:35:11.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.405+0000 7f47a17fa700 1 -- 172.21.15.181:0/3690890584 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479800f460 con 0x7f47a4104130 2024-05-03T20:35:11.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.405+0000 7f47aa843700 1 -- 172.21.15.181:0/3690890584 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a41a6600 con 0x7f47a4104130 2024-05-03T20:35:11.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.406+0000 7f47aa843700 1 -- 172.21.15.181:0/3690890584 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a406a190 con 0x7f47a4104130 2024-05-03T20:35:11.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.407+0000 7f47a17fa700 1 -- 172.21.15.181:0/3690890584 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f479802a070 con 0x7f47a4104130 2024-05-03T20:35:11.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.408+0000 7f47a17fa700 1 --2- 172.21.15.181:0/3690890584 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f478c0b2910 0x7f478c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:11.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.408+0000 7f47a3fff700 1 --2- 172.21.15.181:0/3690890584 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f478c0b2910 0x7f478c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:11.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.408+0000 7f47a17fa700 1 -- 172.21.15.181:0/3690890584 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f47980e3cb0 con 0x7f47a4104130 2024-05-03T20:35:11.422 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.408+0000 7f47a3fff700 1 --2- 172.21.15.181:0/3690890584 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f478c0b2910 0x7f478c0b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4794005fd0 tx=0x7f4794005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:11.422 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.411+0000 7f47a17fa700 1 -- 172.21.15.181:0/3690890584 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4798031020 con 0x7f47a4104130 2024-05-03T20:35:11.624 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.625+0000 7f47aa843700 1 -- 172.21.15.181:0/3690890584 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f47a402e510 con 0x7f47a4104130 2024-05-03T20:35:11.624 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.625+0000 7f47a17fa700 1 -- 172.21.15.181:0/3690890584 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4798013070 con 0x7f47a4104130 2024-05-03T20:35:11.625 INFO:teuthology.orchestra.run.smithi181.stdout:77309411406 2024-05-03T20:35:11.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.627+0000 7f47aa843700 1 -- 172.21.15.181:0/3690890584 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f478c0b2910 msgr2=0x7f478c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:11.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.627+0000 7f47aa843700 1 --2- 172.21.15.181:0/3690890584 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f478c0b2910 0x7f478c0b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4794005fd0 tx=0x7f4794005e40 comp rx=0 tx=0).stop 2024-05-03T20:35:11.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.627+0000 7f47aa843700 1 -- 172.21.15.181:0/3690890584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47a4104130 msgr2=0x7f47a419fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:11.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.627+0000 7f47aa843700 1 --2- 172.21.15.181:0/3690890584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47a4104130 0x7f47a419fc70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f47980098e0 tx=0x7f4798003680 comp rx=0 tx=0).stop 2024-05-03T20:35:11.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.627+0000 7f47aa843700 1 -- 172.21.15.181:0/3690890584 shutdown_connections 2024-05-03T20:35:11.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.627+0000 7f47aa843700 1 --2- 172.21.15.181:0/3690890584 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f478c0b2910 0x7f478c0b4dd0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:11.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.627+0000 7f47aa843700 1 --2- 172.21.15.181:0/3690890584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47a4108870 0x7f47a41998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:11.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.627+0000 7f47aa843700 1 --2- 172.21.15.181:0/3690890584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47a4103780 0x7f47a419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:11.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.627+0000 7f47aa843700 1 --2- 172.21.15.181:0/3690890584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47a4104130 0x7f47a419fc70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:11.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.627+0000 7f47aa843700 1 -- 172.21.15.181:0/3690890584 >> 172.21.15.181:0/3690890584 conn(0x7f47a407a610 msgr2=0x7f47a4102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:11.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.628+0000 7f47aa843700 1 -- 172.21.15.181:0/3690890584 shutdown_connections 2024-05-03T20:35:11.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:11.628+0000 7f47aa843700 1 -- 172.21.15.181:0/3690890584 wait complete. 2024-05-03T20:35:12.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:12 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3352977741' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-03T20:35:12.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:12 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3690890584' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-03T20:35:12.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:12 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3352977741' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-03T20:35:12.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:12 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3690890584' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-03T20:35:12.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:12 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3352977741' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-03T20:35:12.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:12 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3690890584' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-03T20:35:12.605 INFO:teuthology.orchestra.run.smithi181.stdout:34359738454 2024-05-03T20:35:12.606 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-03T20:35:13.026 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972393 got 266287972396 for osd.9 2024-05-03T20:35:13.027 DEBUG:teuthology.parallel:result is None 2024-05-03T20:35:13.141 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:13 smithi181 ceph-mon[29801]: pgmap v332: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:35:13.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:13 smithi196 ceph-mon[32858]: pgmap v332: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:35:13.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:13 smithi204 ceph-mon[33595]: pgmap v332: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:35:14.045 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:35:14.045 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:35:14.574 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411402 got 77309411406 for osd.2 2024-05-03T20:35:14.575 DEBUG:teuthology.parallel:result is None 2024-05-03T20:35:15.196 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:15 smithi181 ceph-mon[29801]: pgmap v333: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:35:15.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.299+0000 7f91bd357700 1 -- 172.21.15.181:0/2251724005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91b8102d60 msgr2=0x7f91b810f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:15.339 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.299+0000 7f91bd357700 1 --2- 172.21.15.181:0/2251724005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91b8102d60 0x7f91b810f250 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f91b000b0a0 tx=0x7f91b000b3b0 comp rx=0 tx=0).stop 2024-05-03T20:35:15.339 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.299+0000 7f91bd357700 1 -- 172.21.15.181:0/2251724005 shutdown_connections 2024-05-03T20:35:15.339 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.299+0000 7f91bd357700 1 --2- 172.21.15.181:0/2251724005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91b8102d60 0x7f91b810f250 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:15.339 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.299+0000 7f91bd357700 1 --2- 172.21.15.181:0/2251724005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91b8108410 0x7f91b81087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:15.339 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.299+0000 7f91bd357700 1 --2- 172.21.15.181:0/2251724005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91b81023a0 0x7f91b8102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:15.339 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.299+0000 7f91bd357700 1 -- 172.21.15.181:0/2251724005 >> 172.21.15.181:0/2251724005 conn(0x7f91b8079350 msgr2=0x7f91b8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:15.339 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.299+0000 7f91bd357700 1 -- 172.21.15.181:0/2251724005 shutdown_connections 2024-05-03T20:35:15.339 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.299+0000 7f91bd357700 1 -- 172.21.15.181:0/2251724005 wait complete. 2024-05-03T20:35:15.339 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.300+0000 7f91bd357700 1 Processor -- start 2024-05-03T20:35:15.339 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.300+0000 7f91bd357700 1 -- start start 2024-05-03T20:35:15.339 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.300+0000 7f91bd357700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91b81023a0 0x7f91b819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:15.339 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.300+0000 7f91bd357700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91b8102d60 0x7f91b819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:15.339 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.300+0000 7f91bd357700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91b8108410 0x7f91b81989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:15.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.300+0000 7f91bd357700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f91b8112630 con 0x7f91b81023a0 2024-05-03T20:35:15.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.300+0000 7f91bd357700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f91b81124b0 con 0x7f91b8108410 2024-05-03T20:35:15.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.300+0000 7f91bd357700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f91b81127b0 con 0x7f91b8102d60 2024-05-03T20:35:15.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.300+0000 7f91b6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91b81023a0 0x7f91b819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:15.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.300+0000 7f91b6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91b81023a0 0x7f91b819e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33940/0 (socket says 172.21.15.181:33940) 2024-05-03T20:35:15.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.300+0000 7f91b6ffd700 1 -- 172.21.15.181:0/4253428670 learned_addr learned my addr 172.21.15.181:0/4253428670 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:15.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.301+0000 7f91b77fe700 1 --2- 172.21.15.181:0/4253428670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91b8108410 0x7f91b81989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:15.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.301+0000 7f91b6ffd700 1 -- 172.21.15.181:0/4253428670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91b8102d60 msgr2=0x7f91b819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:15.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.301+0000 7f91b67fc700 1 --2- 172.21.15.181:0/4253428670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91b8102d60 0x7f91b819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:15.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.301+0000 7f91b6ffd700 1 --2- 172.21.15.181:0/4253428670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91b8102d60 0x7f91b819ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:15.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.301+0000 7f91b6ffd700 1 -- 172.21.15.181:0/4253428670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91b8108410 msgr2=0x7f91b81989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:15.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.301+0000 7f91b6ffd700 1 --2- 172.21.15.181:0/4253428670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91b8108410 0x7f91b81989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:15.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.301+0000 7f91b6ffd700 1 -- 172.21.15.181:0/4253428670 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b8199110 con 0x7f91b81023a0 2024-05-03T20:35:15.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.301+0000 7f91b77fe700 1 --2- 172.21.15.181:0/4253428670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91b8108410 0x7f91b81989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:35:15.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.301+0000 7f91b6ffd700 1 --2- 172.21.15.181:0/4253428670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91b81023a0 0x7f91b819e7f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f91a800b530 tx=0x7f91a800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:15.341 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.301+0000 7f919ffff700 1 -- 172.21.15.181:0/4253428670 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a8015020 con 0x7f91b81023a0 2024-05-03T20:35:15.341 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.301+0000 7f91bd357700 1 -- 172.21.15.181:0/4253428670 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b8199400 con 0x7f91b81023a0 2024-05-03T20:35:15.341 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.301+0000 7f919ffff700 1 -- 172.21.15.181:0/4253428670 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91a8011ce0 con 0x7f91b81023a0 2024-05-03T20:35:15.341 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.301+0000 7f919ffff700 1 -- 172.21.15.181:0/4253428670 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a8010d20 con 0x7f91b81023a0 2024-05-03T20:35:15.341 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.301+0000 7f91bd357700 1 -- 172.21.15.181:0/4253428670 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b81a56c0 con 0x7f91b81023a0 2024-05-03T20:35:15.341 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.302+0000 7f91bd357700 1 -- 172.21.15.181:0/4253428670 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b810a980 con 0x7f91b81023a0 2024-05-03T20:35:15.341 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.303+0000 7f919ffff700 1 -- 172.21.15.181:0/4253428670 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f91a8018020 con 0x7f91b81023a0 2024-05-03T20:35:15.341 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.304+0000 7f919ffff700 1 --2- 172.21.15.181:0/4253428670 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f91a00b2910 0x7f91a00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:15.341 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.304+0000 7f91b67fc700 1 --2- 172.21.15.181:0/4253428670 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f91a00b2910 0x7f91a00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:15.341 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.304+0000 7f919ffff700 1 -- 172.21.15.181:0/4253428670 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f91a80e1fa0 con 0x7f91b81023a0 2024-05-03T20:35:15.341 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.304+0000 7f91b67fc700 1 --2- 172.21.15.181:0/4253428670 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f91a00b2910 0x7f91a00b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f91ac006fd0 tx=0x7f91ac008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:15.341 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.308+0000 7f919ffff700 1 -- 172.21.15.181:0/4253428670 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f91a801f020 con 0x7f91b81023a0 2024-05-03T20:35:15.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:15 smithi196 ceph-mon[32858]: pgmap v333: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:35:15.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:15 smithi204 ceph-mon[33595]: pgmap v333: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:35:15.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.534+0000 7f91bd357700 1 -- 172.21.15.181:0/4253428670 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f91b8199ee0 con 0x7f91b81023a0 2024-05-03T20:35:15.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.535+0000 7f919ffff700 1 -- 172.21.15.181:0/4253428670 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f91a80a9d10 con 0x7f91b81023a0 2024-05-03T20:35:15.534 INFO:teuthology.orchestra.run.smithi181.stdout:287762808871 2024-05-03T20:35:15.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.537+0000 7f91bd357700 1 -- 172.21.15.181:0/4253428670 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f91a00b2910 msgr2=0x7f91a00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:15.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.537+0000 7f91bd357700 1 --2- 172.21.15.181:0/4253428670 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f91a00b2910 0x7f91a00b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f91ac006fd0 tx=0x7f91ac008040 comp rx=0 tx=0).stop 2024-05-03T20:35:15.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.537+0000 7f91bd357700 1 -- 172.21.15.181:0/4253428670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91b81023a0 msgr2=0x7f91b819e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:15.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.537+0000 7f91bd357700 1 --2- 172.21.15.181:0/4253428670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91b81023a0 0x7f91b819e7f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f91a800b530 tx=0x7f91a800b8f0 comp rx=0 tx=0).stop 2024-05-03T20:35:15.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.537+0000 7f91bd357700 1 -- 172.21.15.181:0/4253428670 shutdown_connections 2024-05-03T20:35:15.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.537+0000 7f91bd357700 1 --2- 172.21.15.181:0/4253428670 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f91a00b2910 0x7f91a00b4dd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:15.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.537+0000 7f91bd357700 1 --2- 172.21.15.181:0/4253428670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91b8108410 0x7f91b81989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:15.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.537+0000 7f91bd357700 1 --2- 172.21.15.181:0/4253428670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91b81023a0 0x7f91b819e7f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:15.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.537+0000 7f91bd357700 1 --2- 172.21.15.181:0/4253428670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91b8102d60 0x7f91b819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:15.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.537+0000 7f91bd357700 1 -- 172.21.15.181:0/4253428670 >> 172.21.15.181:0/4253428670 conn(0x7f91b8079350 msgr2=0x7f91b8101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:15.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.537+0000 7f91bd357700 1 -- 172.21.15.181:0/4253428670 shutdown_connections 2024-05-03T20:35:15.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:15.537+0000 7f91bd357700 1 -- 172.21.15.181:0/4253428670 wait complete. 2024-05-03T20:35:16.353 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:16 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/4253428670' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-03T20:35:16.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:16 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/4253428670' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-03T20:35:16.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:16 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/4253428670' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-03T20:35:17.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.021+0000 7f4741915700 1 -- 172.21.15.181:0/1987671994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f473c06bc10 msgr2=0x7f473c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:17.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.021+0000 7f4741915700 1 --2- 172.21.15.181:0/1987671994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f473c06bc10 0x7f473c06c090 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4730009990 tx=0x7f4730009ca0 comp rx=0 tx=0).stop 2024-05-03T20:35:17.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.021+0000 7f4741915700 1 -- 172.21.15.181:0/1987671994 shutdown_connections 2024-05-03T20:35:17.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.021+0000 7f4741915700 1 --2- 172.21.15.181:0/1987671994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f473c06c5d0 0x7f473c110370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:17.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.021+0000 7f4741915700 1 --2- 172.21.15.181:0/1987671994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f473c109580 0x7f473c109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:17.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.021+0000 7f4741915700 1 --2- 172.21.15.181:0/1987671994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f473c06bc10 0x7f473c06c090 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:17.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.021+0000 7f4741915700 1 -- 172.21.15.181:0/1987671994 >> 172.21.15.181:0/1987671994 conn(0x7f473c0783f0 msgr2=0x7f473c078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:17.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.021+0000 7f4741915700 1 -- 172.21.15.181:0/1987671994 shutdown_connections 2024-05-03T20:35:17.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.021+0000 7f4741915700 1 -- 172.21.15.181:0/1987671994 wait complete. 2024-05-03T20:35:17.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.022+0000 7f4741915700 1 Processor -- start 2024-05-03T20:35:17.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.022+0000 7f4741915700 1 -- start start 2024-05-03T20:35:17.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.022+0000 7f4741915700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f473c06bc10 0x7f473c19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:17.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.022+0000 7f4741915700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f473c06c5d0 0x7f473c19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:17.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.022+0000 7f4741915700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f473c109580 0x7f473c199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:17.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.022+0000 7f4741915700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f473c1136b0 con 0x7f473c06c5d0 2024-05-03T20:35:17.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.022+0000 7f4741915700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f473c113530 con 0x7f473c06bc10 2024-05-03T20:35:17.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.022+0000 7f4741915700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f473c113830 con 0x7f473c109580 2024-05-03T20:35:17.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.022+0000 7f473a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f473c06c5d0 0x7f473c19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:17.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.022+0000 7f473a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f473c06c5d0 0x7f473c19fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50574/0 (socket says 172.21.15.181:50574) 2024-05-03T20:35:17.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.022+0000 7f473a7fc700 1 -- 172.21.15.181:0/1712071513 learned_addr learned my addr 172.21.15.181:0/1712071513 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:17.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.022+0000 7f473affd700 1 --2- 172.21.15.181:0/1712071513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f473c06bc10 0x7f473c19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:17.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.023+0000 7f473a7fc700 1 -- 172.21.15.181:0/1712071513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f473c109580 msgr2=0x7f473c199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:17.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.023+0000 7f473a7fc700 1 --2- 172.21.15.181:0/1712071513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f473c109580 0x7f473c199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:17.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.023+0000 7f473a7fc700 1 -- 172.21.15.181:0/1712071513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f473c06bc10 msgr2=0x7f473c19f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:17.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.023+0000 7f473a7fc700 1 --2- 172.21.15.181:0/1712071513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f473c06bc10 0x7f473c19f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:17.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.023+0000 7f473a7fc700 1 -- 172.21.15.181:0/1712071513 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f473c19a0d0 con 0x7f473c06c5d0 2024-05-03T20:35:17.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.023+0000 7f473affd700 1 --2- 172.21.15.181:0/1712071513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f473c06bc10 0x7f473c19f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:35:17.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.023+0000 7f473a7fc700 1 --2- 172.21.15.181:0/1712071513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f473c06c5d0 0x7f473c19fcf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4730000c00 tx=0x7f4730005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:17.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.023+0000 7f4740913700 1 -- 172.21.15.181:0/1712071513 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4730027070 con 0x7f473c06c5d0 2024-05-03T20:35:17.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.023+0000 7f4741915700 1 -- 172.21.15.181:0/1712071513 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f473c19a360 con 0x7f473c06c5d0 2024-05-03T20:35:17.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.023+0000 7f4740913700 1 -- 172.21.15.181:0/1712071513 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4730003f20 con 0x7f473c06c5d0 2024-05-03T20:35:17.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.023+0000 7f4740913700 1 -- 172.21.15.181:0/1712071513 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4730022c70 con 0x7f473c06c5d0 2024-05-03T20:35:17.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.023+0000 7f4741915700 1 -- 172.21.15.181:0/1712071513 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f473c1a6680 con 0x7f473c06c5d0 2024-05-03T20:35:17.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.024+0000 7f4741915700 1 -- 172.21.15.181:0/1712071513 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f473c06a190 con 0x7f473c06c5d0 2024-05-03T20:35:17.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.025+0000 7f4740913700 1 -- 172.21.15.181:0/1712071513 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f473002a070 con 0x7f473c06c5d0 2024-05-03T20:35:17.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.026+0000 7f4740913700 1 --2- 172.21.15.181:0/1712071513 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f47240b2910 0x7f47240b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:17.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.026+0000 7f473affd700 1 --2- 172.21.15.181:0/1712071513 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f47240b2910 0x7f47240b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:17.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.026+0000 7f4740913700 1 -- 172.21.15.181:0/1712071513 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f47300e30d0 con 0x7f473c06c5d0 2024-05-03T20:35:17.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.026+0000 7f473affd700 1 --2- 172.21.15.181:0/1712071513 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f47240b2910 0x7f47240b4dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f472c005950 tx=0x7f472c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:17.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.030+0000 7f4740913700 1 -- 172.21.15.181:0/1712071513 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4730013070 con 0x7f473c06c5d0 2024-05-03T20:35:17.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.219+0000 7fc06cb5f700 1 -- 172.21.15.181:0/3526931228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc06806bb40 msgr2=0x7fc068110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:17.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.219+0000 7fc06cb5f700 1 --2- 172.21.15.181:0/3526931228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc06806bb40 0x7fc068110330 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc06000b0a0 tx=0x7fc06000b3b0 comp rx=0 tx=0).stop 2024-05-03T20:35:17.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.219+0000 7fc06cb5f700 1 -- 172.21.15.181:0/3526931228 shutdown_connections 2024-05-03T20:35:17.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.219+0000 7fc06cb5f700 1 --2- 172.21.15.181:0/3526931228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc06806bb40 0x7fc068110330 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:17.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.219+0000 7fc06cb5f700 1 --2- 172.21.15.181:0/3526931228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc068109510 0x7fc0681098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:17.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.219+0000 7fc06cb5f700 1 --2- 172.21.15.181:0/3526931228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc06806b180 0x7fc06806b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:17.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.219+0000 7fc06cb5f700 1 -- 172.21.15.181:0/3526931228 >> 172.21.15.181:0/3526931228 conn(0x7fc068079350 msgr2=0x7fc068079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:17.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.219+0000 7fc06cb5f700 1 -- 172.21.15.181:0/3526931228 shutdown_connections 2024-05-03T20:35:17.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.219+0000 7fc06cb5f700 1 -- 172.21.15.181:0/3526931228 wait complete. 2024-05-03T20:35:17.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.220+0000 7fc06cb5f700 1 Processor -- start 2024-05-03T20:35:17.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.220+0000 7fc06cb5f700 1 -- start start 2024-05-03T20:35:17.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.220+0000 7fc06cb5f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc06806b180 0x7fc068102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:17.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.220+0000 7fc06cb5f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc06806bb40 0x7fc0681029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:17.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.220+0000 7fc06cb5f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc068109510 0x7fc068102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:17.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.220+0000 7fc06cb5f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc0681136a0 con 0x7fc068109510 2024-05-03T20:35:17.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.220+0000 7fc06cb5f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc068113520 con 0x7fc06806b180 2024-05-03T20:35:17.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.220+0000 7fc06cb5f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc068113820 con 0x7fc06806bb40 2024-05-03T20:35:17.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.220+0000 7fc06659c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc06806b180 0x7fc068102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:17.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.221+0000 7fc06659c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc06806b180 0x7fc068102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.181:55046/0 (socket says 172.21.15.181:55046) 2024-05-03T20:35:17.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.221+0000 7fc06659c700 1 -- 172.21.15.181:0/3443333140 learned_addr learned my addr 172.21.15.181:0/3443333140 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:17.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.221+0000 7fc065d9b700 1 --2- 172.21.15.181:0/3443333140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc06806bb40 0x7fc0681029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:17.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.221+0000 7fc066d9d700 1 --2- 172.21.15.181:0/3443333140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc068109510 0x7fc068102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:17.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.221+0000 7fc06659c700 1 -- 172.21.15.181:0/3443333140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc06806bb40 msgr2=0x7fc0681029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:17.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.221+0000 7fc06659c700 1 --2- 172.21.15.181:0/3443333140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc06806bb40 0x7fc0681029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:17.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.221+0000 7fc06659c700 1 -- 172.21.15.181:0/3443333140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc068109510 msgr2=0x7fc068102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:17.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.221+0000 7fc06659c700 1 --2- 172.21.15.181:0/3443333140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc068109510 0x7fc068102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:17.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.221+0000 7fc06659c700 1 -- 172.21.15.181:0/3443333140 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0681a80a0 con 0x7fc06806b180 2024-05-03T20:35:17.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.221+0000 7fc06659c700 1 --2- 172.21.15.181:0/3443333140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc06806b180 0x7fc068102470 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc05800b530 tx=0x7fc05800b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:17.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.221+0000 7fc0577fe700 1 -- 172.21.15.181:0/3443333140 <== mon.2 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058015020 con 0x7fc06806b180 2024-05-03T20:35:17.224 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.221+0000 7fc0577fe700 1 -- 172.21.15.181:0/3443333140 <== mon.2 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc058011ce0 con 0x7fc06806b180 2024-05-03T20:35:17.224 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.221+0000 7fc06cb5f700 1 -- 172.21.15.181:0/3443333140 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0681a8390 con 0x7fc06806b180 2024-05-03T20:35:17.224 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.221+0000 7fc0577fe700 1 -- 172.21.15.181:0/3443333140 <== mon.2 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058010dd0 con 0x7fc06806b180 2024-05-03T20:35:17.224 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.221+0000 7fc06cb5f700 1 -- 172.21.15.181:0/3443333140 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0681a88d0 con 0x7fc06806b180 2024-05-03T20:35:17.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.222+0000 7fc06cb5f700 1 -- 172.21.15.181:0/3443333140 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc06810b9f0 con 0x7fc06806b180 2024-05-03T20:35:17.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.224+0000 7fc0577fe700 1 -- 172.21.15.181:0/3443333140 <== mon.2 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fc058018020 con 0x7fc06806b180 2024-05-03T20:35:17.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.224+0000 7fc0577fe700 1 --2- 172.21.15.181:0/3443333140 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fc0500b2910 0x7fc0500b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:17.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.224+0000 7fc065d9b700 1 --2- 172.21.15.181:0/3443333140 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fc0500b2910 0x7fc0500b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:17.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.224+0000 7fc0577fe700 1 -- 172.21.15.181:0/3443333140 <== mon.2 v2:172.21.15.196:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7fc0580e1ff0 con 0x7fc06806b180 2024-05-03T20:35:17.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.225+0000 7fc065d9b700 1 --2- 172.21.15.181:0/3443333140 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fc0500b2910 0x7fc0500b4dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc068079950 tx=0x7fc05c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:17.227 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.228+0000 7fc0577fe700 1 -- 172.21.15.181:0/3443333140 <== mon.2 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc05801f020 con 0x7fc06806b180 2024-05-03T20:35:17.242 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:35:17.246 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.246+0000 7f4741915700 1 -- 172.21.15.181:0/1712071513 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f473c10b980 con 0x7f473c06c5d0 2024-05-03T20:35:17.246 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.246+0000 7f4740913700 1 -- 172.21.15.181:0/1712071513 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f47300aae30 con 0x7f473c06c5d0 2024-05-03T20:35:17.247 INFO:teuthology.orchestra.run.smithi181.stdout:163208757312 2024-05-03T20:35:17.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.249+0000 7f4741915700 1 -- 172.21.15.181:0/1712071513 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f47240b2910 msgr2=0x7f47240b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:17.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.249+0000 7f4741915700 1 --2- 172.21.15.181:0/1712071513 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f47240b2910 0x7f47240b4dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f472c005950 tx=0x7f472c0058e0 comp rx=0 tx=0).stop 2024-05-03T20:35:17.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.249+0000 7f4741915700 1 -- 172.21.15.181:0/1712071513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f473c06c5d0 msgr2=0x7f473c19fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:17.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.249+0000 7f4741915700 1 --2- 172.21.15.181:0/1712071513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f473c06c5d0 0x7f473c19fcf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4730000c00 tx=0x7f4730005ef0 comp rx=0 tx=0).stop 2024-05-03T20:35:17.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.249+0000 7f4741915700 1 -- 172.21.15.181:0/1712071513 shutdown_connections 2024-05-03T20:35:17.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.249+0000 7f4741915700 1 --2- 172.21.15.181:0/1712071513 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f47240b2910 0x7f47240b4dd0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:17.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.249+0000 7f4741915700 1 --2- 172.21.15.181:0/1712071513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f473c109580 0x7f473c199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:17.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.249+0000 7f4741915700 1 --2- 172.21.15.181:0/1712071513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f473c06bc10 0x7f473c19f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:17.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.249+0000 7f4741915700 1 --2- 172.21.15.181:0/1712071513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f473c06c5d0 0x7f473c19fcf0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:17.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.249+0000 7f4741915700 1 -- 172.21.15.181:0/1712071513 >> 172.21.15.181:0/1712071513 conn(0x7f473c0783f0 msgr2=0x7f473c101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:17.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.249+0000 7f4741915700 1 -- 172.21.15.181:0/1712071513 shutdown_connections 2024-05-03T20:35:17.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.249+0000 7f4741915700 1 -- 172.21.15.181:0/1712071513 wait complete. 2024-05-03T20:35:17.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:17 smithi196 ceph-mon[32858]: pgmap v334: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:35:17.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:17 smithi204 ceph-mon[33595]: pgmap v334: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:35:17.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.453+0000 7fc06cb5f700 1 -- 172.21.15.181:0/3443333140 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fc06810b3d0 con 0x7fc06806b180 2024-05-03T20:35:17.453 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:17 smithi181 ceph-mon[29801]: pgmap v334: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:35:17.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.453+0000 7fc0577fe700 1 -- 172.21.15.181:0/3443333140 <== mon.2 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc0580a9c60 con 0x7fc06806b180 2024-05-03T20:35:17.453 INFO:teuthology.orchestra.run.smithi181.stdout:141733920836 2024-05-03T20:35:17.455 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.456+0000 7fc06cb5f700 1 -- 172.21.15.181:0/3443333140 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fc0500b2910 msgr2=0x7fc0500b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:17.455 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.456+0000 7fc06cb5f700 1 --2- 172.21.15.181:0/3443333140 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fc0500b2910 0x7fc0500b4dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc068079950 tx=0x7fc05c00a300 comp rx=0 tx=0).stop 2024-05-03T20:35:17.455 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.456+0000 7fc06cb5f700 1 -- 172.21.15.181:0/3443333140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc06806b180 msgr2=0x7fc068102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:17.455 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.456+0000 7fc06cb5f700 1 --2- 172.21.15.181:0/3443333140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc06806b180 0x7fc068102470 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc05800b530 tx=0x7fc05800b840 comp rx=0 tx=0).stop 2024-05-03T20:35:17.455 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.456+0000 7fc06cb5f700 1 -- 172.21.15.181:0/3443333140 shutdown_connections 2024-05-03T20:35:17.456 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.456+0000 7fc06cb5f700 1 --2- 172.21.15.181:0/3443333140 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fc0500b2910 0x7fc0500b4dd0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:17.456 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.456+0000 7fc06cb5f700 1 --2- 172.21.15.181:0/3443333140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc068109510 0x7fc068102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:17.456 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.456+0000 7fc06cb5f700 1 --2- 172.21.15.181:0/3443333140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc06806b180 0x7fc068102470 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:17.456 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.456+0000 7fc06cb5f700 1 --2- 172.21.15.181:0/3443333140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc06806bb40 0x7fc0681029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:17.456 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.456+0000 7fc06cb5f700 1 -- 172.21.15.181:0/3443333140 >> 172.21.15.181:0/3443333140 conn(0x7fc068079350 msgr2=0x7fc068100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:17.456 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.456+0000 7fc06cb5f700 1 -- 172.21.15.181:0/3443333140 shutdown_connections 2024-05-03T20:35:17.456 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:17.456+0000 7fc06cb5f700 1 -- 172.21.15.181:0/3443333140 wait complete. 2024-05-03T20:35:18.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:18 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/1712071513' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-03T20:35:18.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:18 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3443333140' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-03T20:35:18.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:18 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/1712071513' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-03T20:35:18.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:18 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3443333140' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-03T20:35:18.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:18 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/1712071513' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-03T20:35:18.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:18 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3443333140' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-03T20:35:18.570 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.570+0000 7f592e57d700 1 -- 172.21.15.181:0/753897556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59281095e0 msgr2=0x7f59281099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:18.570 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.570+0000 7f592e57d700 1 --2- 172.21.15.181:0/753897556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59281095e0 0x7f59281099c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f59180099c0 tx=0x7f5918009cd0 comp rx=0 tx=0).stop 2024-05-03T20:35:18.570 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.570+0000 7f592e57d700 1 -- 172.21.15.181:0/753897556 shutdown_connections 2024-05-03T20:35:18.570 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.570+0000 7f592e57d700 1 --2- 172.21.15.181:0/753897556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5928103f00 0x7f59281103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:18.570 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.570+0000 7f592e57d700 1 --2- 172.21.15.181:0/753897556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59281095e0 0x7f59281099c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:18.570 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.570+0000 7f592e57d700 1 --2- 172.21.15.181:0/753897556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5928103540 0x7f59281039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:18.570 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.570+0000 7f592e57d700 1 -- 172.21.15.181:0/753897556 >> 172.21.15.181:0/753897556 conn(0x7f592807a650 msgr2=0x7f592807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:18.571 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.570+0000 7f592e57d700 1 -- 172.21.15.181:0/753897556 shutdown_connections 2024-05-03T20:35:18.571 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.570+0000 7f592e57d700 1 -- 172.21.15.181:0/753897556 wait complete. 2024-05-03T20:35:18.571 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.571+0000 7f592e57d700 1 Processor -- start 2024-05-03T20:35:18.571 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.571+0000 7f592e57d700 1 -- start start 2024-05-03T20:35:18.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.571+0000 7f592e57d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5928103540 0x7f59280757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:18.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.571+0000 7f592e57d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5928103f00 0x7f5928075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:18.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.571+0000 7f592e57d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59281095e0 0x7f5928076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:18.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.571+0000 7f592e57d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5928113750 con 0x7f59281095e0 2024-05-03T20:35:18.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.571+0000 7f592e57d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f59281135d0 con 0x7f5928103f00 2024-05-03T20:35:18.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.571+0000 7f592e57d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f59281138d0 con 0x7f5928103540 2024-05-03T20:35:18.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.571+0000 7f592cb1a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59281095e0 0x7f5928076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:18.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.571+0000 7f592cb1a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59281095e0 0x7f5928076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50604/0 (socket says 172.21.15.181:50604) 2024-05-03T20:35:18.574 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.571+0000 7f592cb1a700 1 -- 172.21.15.181:0/364634509 learned_addr learned my addr 172.21.15.181:0/364634509 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:18.574 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.571+0000 7f59277fe700 1 --2- 172.21.15.181:0/364634509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5928103f00 0x7f5928075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:18.574 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.571+0000 7f5927fff700 1 --2- 172.21.15.181:0/364634509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5928103540 0x7f59280757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:18.574 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.571+0000 7f592cb1a700 1 -- 172.21.15.181:0/364634509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5928103540 msgr2=0x7f59280757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:18.574 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.571+0000 7f592cb1a700 1 --2- 172.21.15.181:0/364634509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5928103540 0x7f59280757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:18.574 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.571+0000 7f592cb1a700 1 -- 172.21.15.181:0/364634509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5928103f00 msgr2=0x7f5928075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:18.574 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.571+0000 7f592cb1a700 1 --2- 172.21.15.181:0/364634509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5928103f00 0x7f5928075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:18.574 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.571+0000 7f592cb1a700 1 -- 172.21.15.181:0/364634509 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59281a8170 con 0x7f59281095e0 2024-05-03T20:35:18.575 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.572+0000 7f592cb1a700 1 --2- 172.21.15.181:0/364634509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59281095e0 0x7f5928076220 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5920007ee0 tx=0x7f592000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:18.575 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.572+0000 7f59257fa700 1 -- 172.21.15.181:0/364634509 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592000ddd0 con 0x7f59281095e0 2024-05-03T20:35:18.575 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.572+0000 7f592e57d700 1 -- 172.21.15.181:0/364634509 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59281a8460 con 0x7f59281095e0 2024-05-03T20:35:18.575 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.572+0000 7f59257fa700 1 -- 172.21.15.181:0/364634509 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5920004020 con 0x7f59281095e0 2024-05-03T20:35:18.575 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.572+0000 7f59257fa700 1 -- 172.21.15.181:0/364634509 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920015660 con 0x7f59281095e0 2024-05-03T20:35:18.575 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.572+0000 7f592e57d700 1 -- 172.21.15.181:0/364634509 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59281a8950 con 0x7f59281095e0 2024-05-03T20:35:18.575 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.573+0000 7f592e57d700 1 -- 172.21.15.181:0/364634509 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f592810ba70 con 0x7f59281095e0 2024-05-03T20:35:18.576 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.574+0000 7f59257fa700 1 -- 172.21.15.181:0/364634509 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f59200041d0 con 0x7f59281095e0 2024-05-03T20:35:18.576 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.575+0000 7f59257fa700 1 --2- 172.21.15.181:0/364634509 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f59100b2910 0x7f59100b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:18.576 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.575+0000 7f5927fff700 1 --2- 172.21.15.181:0/364634509 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f59100b2910 0x7f59100b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:18.577 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.575+0000 7f59257fa700 1 -- 172.21.15.181:0/364634509 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f59200e25b0 con 0x7f59281095e0 2024-05-03T20:35:18.577 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.575+0000 7f5927fff700 1 --2- 172.21.15.181:0/364634509 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f59100b2910 0x7f59100b4dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5918003800 tx=0x7f591801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:18.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.578+0000 7f59257fa700 1 -- 172.21.15.181:0/364634509 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5920010070 con 0x7f59281095e0 2024-05-03T20:35:18.786 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.786+0000 7f592e57d700 1 -- 172.21.15.181:0/364634509 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f5928076cd0 con 0x7f59281095e0 2024-05-03T20:35:18.786 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.786+0000 7f59257fa700 1 -- 172.21.15.181:0/364634509 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f59200aa170 con 0x7f59281095e0 2024-05-03T20:35:18.786 INFO:teuthology.orchestra.run.smithi181.stdout:115964117066 2024-05-03T20:35:18.788 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.789+0000 7f592e57d700 1 -- 172.21.15.181:0/364634509 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f59100b2910 msgr2=0x7f59100b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:18.788 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.789+0000 7f592e57d700 1 --2- 172.21.15.181:0/364634509 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f59100b2910 0x7f59100b4dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5918003800 tx=0x7f591801a040 comp rx=0 tx=0).stop 2024-05-03T20:35:18.788 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.789+0000 7f592e57d700 1 -- 172.21.15.181:0/364634509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59281095e0 msgr2=0x7f5928076220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:18.789 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.789+0000 7f592e57d700 1 --2- 172.21.15.181:0/364634509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59281095e0 0x7f5928076220 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5920007ee0 tx=0x7f592000d3b0 comp rx=0 tx=0).stop 2024-05-03T20:35:18.789 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.789+0000 7f592e57d700 1 -- 172.21.15.181:0/364634509 shutdown_connections 2024-05-03T20:35:18.789 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.789+0000 7f592e57d700 1 --2- 172.21.15.181:0/364634509 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f59100b2910 0x7f59100b4dd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:18.789 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.789+0000 7f592e57d700 1 --2- 172.21.15.181:0/364634509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59281095e0 0x7f5928076220 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:18.789 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.789+0000 7f592e57d700 1 --2- 172.21.15.181:0/364634509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5928103540 0x7f59280757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:18.789 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.789+0000 7f592e57d700 1 --2- 172.21.15.181:0/364634509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5928103f00 0x7f5928075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:18.789 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.789+0000 7f592e57d700 1 -- 172.21.15.181:0/364634509 >> 172.21.15.181:0/364634509 conn(0x7f592807a650 msgr2=0x7f5928102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:18.789 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.789+0000 7f592e57d700 1 -- 172.21.15.181:0/364634509 shutdown_connections 2024-05-03T20:35:18.789 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:18.789+0000 7f592e57d700 1 -- 172.21.15.181:0/364634509 wait complete. 2024-05-03T20:35:19.274 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:19 smithi181 ceph-mon[29801]: pgmap v335: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:35:19.274 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:19 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/364634509' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-03T20:35:19.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:19 smithi196 ceph-mon[32858]: pgmap v335: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:35:19.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:19 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/364634509' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-03T20:35:19.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:19 smithi204 ceph-mon[33595]: pgmap v335: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:35:19.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:19 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/364634509' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-03T20:35:19.561 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117061 got 115964117066 for osd.3 2024-05-03T20:35:19.561 DEBUG:teuthology.parallel:result is None 2024-05-03T20:35:19.571 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:35:21.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:21 smithi196 ceph-mon[32858]: pgmap v336: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:35:21.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:21 smithi204 ceph-mon[33595]: pgmap v336: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:35:21.422 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:21 smithi181 ceph-mon[29801]: pgmap v336: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:35:21.791 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808867 got 287762808871 for osd.10 2024-05-03T20:35:21.791 DEBUG:teuthology.parallel:result is None 2024-05-03T20:35:23.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:23 smithi181 ceph-mon[29801]: pgmap v337: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:35:23.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:23 smithi196 ceph-mon[32858]: pgmap v337: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:35:23.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:23 smithi204 ceph-mon[33595]: pgmap v337: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:35:23.415 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757308 got 163208757312 for osd.5 2024-05-03T20:35:23.415 DEBUG:teuthology.parallel:result is None 2024-05-03T20:35:23.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.599+0000 7fbd11e93700 1 -- 172.21.15.181:0/3347134846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c108410 msgr2=0x7fbd0c1087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:23.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.599+0000 7fbd11e93700 1 --2- 172.21.15.181:0/3347134846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c108410 0x7fbd0c1087f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc009990 tx=0x7fbcfc009ca0 comp rx=0 tx=0).stop 2024-05-03T20:35:23.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.599+0000 7fbd11e93700 1 -- 172.21.15.181:0/3347134846 shutdown_connections 2024-05-03T20:35:23.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.599+0000 7fbd11e93700 1 --2- 172.21.15.181:0/3347134846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd0c102d60 0x7fbd0c10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:23.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.599+0000 7fbd11e93700 1 --2- 172.21.15.181:0/3347134846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c108410 0x7fbd0c1087f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:23.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.599+0000 7fbd11e93700 1 --2- 172.21.15.181:0/3347134846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd0c1023a0 0x7fbd0c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:23.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.599+0000 7fbd11e93700 1 -- 172.21.15.181:0/3347134846 >> 172.21.15.181:0/3347134846 conn(0x7fbd0c079350 msgr2=0x7fbd0c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:23.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.599+0000 7fbd11e93700 1 -- 172.21.15.181:0/3347134846 shutdown_connections 2024-05-03T20:35:23.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.599+0000 7fbd11e93700 1 -- 172.21.15.181:0/3347134846 wait complete. 2024-05-03T20:35:23.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.600+0000 7fbd11e93700 1 Processor -- start 2024-05-03T20:35:23.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.600+0000 7fbd11e93700 1 -- start start 2024-05-03T20:35:23.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.600+0000 7fbd11e93700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd0c1023a0 0x7fbd0c19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:23.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.600+0000 7fbd11e93700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c102d60 0x7fbd0c19aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:23.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.600+0000 7fbd11e93700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd0c108410 0x7fbd0c1a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:23.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.600+0000 7fbd11e93700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c112530 con 0x7fbd0c102d60 2024-05-03T20:35:23.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.600+0000 7fbd11e93700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c1123b0 con 0x7fbd0c108410 2024-05-03T20:35:23.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.600+0000 7fbd11e93700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c1126b0 con 0x7fbd0c1023a0 2024-05-03T20:35:23.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.600+0000 7fbd0affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c102d60 0x7fbd0c19aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:23.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.600+0000 7fbd0affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c102d60 0x7fbd0c19aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50622/0 (socket says 172.21.15.181:50622) 2024-05-03T20:35:23.602 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.600+0000 7fbd0affd700 1 -- 172.21.15.181:0/3023599196 learned_addr learned my addr 172.21.15.181:0/3023599196 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:23.602 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.601+0000 7fbd0b7fe700 1 --2- 172.21.15.181:0/3023599196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd0c1023a0 0x7fbd0c19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:23.602 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.601+0000 7fbd0bfff700 1 --2- 172.21.15.181:0/3023599196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd0c108410 0x7fbd0c1a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:23.602 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.601+0000 7fbd0affd700 1 -- 172.21.15.181:0/3023599196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd0c1023a0 msgr2=0x7fbd0c19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:23.602 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.601+0000 7fbd0affd700 1 --2- 172.21.15.181:0/3023599196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd0c1023a0 0x7fbd0c19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:23.602 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.601+0000 7fbd0affd700 1 -- 172.21.15.181:0/3023599196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd0c108410 msgr2=0x7fbd0c1a5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:23.602 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.601+0000 7fbd0affd700 1 --2- 172.21.15.181:0/3023599196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd0c108410 0x7fbd0c1a5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:23.602 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.601+0000 7fbd0affd700 1 -- 172.21.15.181:0/3023599196 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0c1a5f70 con 0x7fbd0c102d60 2024-05-03T20:35:23.602 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.601+0000 7fbd0bfff700 1 --2- 172.21.15.181:0/3023599196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd0c108410 0x7fbd0c1a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:35:23.602 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.601+0000 7fbd0affd700 1 --2- 172.21.15.181:0/3023599196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c102d60 0x7fbd0c19aef0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbd0000e9a0 tx=0x7fbd0000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:23.602 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.601+0000 7fbd08ff9700 1 -- 172.21.15.181:0/3023599196 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0000cbd0 con 0x7fbd0c102d60 2024-05-03T20:35:23.602 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.601+0000 7fbd08ff9700 1 -- 172.21.15.181:0/3023599196 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd0000cd70 con 0x7fbd0c102d60 2024-05-03T20:35:23.602 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.601+0000 7fbd08ff9700 1 -- 172.21.15.181:0/3023599196 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd00018680 con 0x7fbd0c102d60 2024-05-03T20:35:23.602 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.601+0000 7fbd11e93700 1 -- 172.21.15.181:0/3023599196 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd0c1a61a0 con 0x7fbd0c102d60 2024-05-03T20:35:23.602 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.601+0000 7fbd11e93700 1 -- 172.21.15.181:0/3023599196 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd0c1a66e0 con 0x7fbd0c102d60 2024-05-03T20:35:23.603 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.602+0000 7fbd11e93700 1 -- 172.21.15.181:0/3023599196 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd0c06a190 con 0x7fbd0c102d60 2024-05-03T20:35:23.603 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.603+0000 7fbd08ff9700 1 -- 172.21.15.181:0/3023599196 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fbd00010970 con 0x7fbd0c102d60 2024-05-03T20:35:23.603 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.604+0000 7fbd08ff9700 1 --2- 172.21.15.181:0/3023599196 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fbcf40b2910 0x7fbcf40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:23.603 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.604+0000 7fbd0b7fe700 1 --2- 172.21.15.181:0/3023599196 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fbcf40b2910 0x7fbcf40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:23.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.604+0000 7fbd08ff9700 1 -- 172.21.15.181:0/3023599196 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7fbd000e2bd0 con 0x7fbd0c102d60 2024-05-03T20:35:23.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.604+0000 7fbd0b7fe700 1 --2- 172.21.15.181:0/3023599196 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fbcf40b2910 0x7fbcf40b4dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc006010 tx=0x7fbcfc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:23.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.608+0000 7fbd08ff9700 1 -- 172.21.15.181:0/3023599196 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fbd000aa790 con 0x7fbd0c102d60 2024-05-03T20:35:23.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.820+0000 7fbd11e93700 1 -- 172.21.15.181:0/3023599196 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fbd0c02e510 con 0x7fbd0c102d60 2024-05-03T20:35:23.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.821+0000 7fbd08ff9700 1 -- 172.21.15.181:0/3023599196 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbd000aa090 con 0x7fbd0c102d60 2024-05-03T20:35:23.821 INFO:teuthology.orchestra.run.smithi181.stdout:188978561085 2024-05-03T20:35:23.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.823+0000 7fbd11e93700 1 -- 172.21.15.181:0/3023599196 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fbcf40b2910 msgr2=0x7fbcf40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:23.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.823+0000 7fbd11e93700 1 --2- 172.21.15.181:0/3023599196 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fbcf40b2910 0x7fbcf40b4dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc006010 tx=0x7fbcfc01a040 comp rx=0 tx=0).stop 2024-05-03T20:35:23.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.823+0000 7fbd11e93700 1 -- 172.21.15.181:0/3023599196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c102d60 msgr2=0x7fbd0c19aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:23.824 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.823+0000 7fbd11e93700 1 --2- 172.21.15.181:0/3023599196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c102d60 0x7fbd0c19aef0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbd0000e9a0 tx=0x7fbd0000ed60 comp rx=0 tx=0).stop 2024-05-03T20:35:23.824 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.823+0000 7fbd11e93700 1 -- 172.21.15.181:0/3023599196 shutdown_connections 2024-05-03T20:35:23.824 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.823+0000 7fbd11e93700 1 --2- 172.21.15.181:0/3023599196 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fbcf40b2910 0x7fbcf40b4dd0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:23.824 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.823+0000 7fbd11e93700 1 --2- 172.21.15.181:0/3023599196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd0c108410 0x7fbd0c1a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:23.824 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.823+0000 7fbd11e93700 1 --2- 172.21.15.181:0/3023599196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd0c1023a0 0x7fbd0c19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:23.824 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.823+0000 7fbd11e93700 1 --2- 172.21.15.181:0/3023599196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c102d60 0x7fbd0c19aef0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:23.824 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.823+0000 7fbd11e93700 1 -- 172.21.15.181:0/3023599196 >> 172.21.15.181:0/3023599196 conn(0x7fbd0c079350 msgr2=0x7fbd0c101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:23.825 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.823+0000 7fbd11e93700 1 -- 172.21.15.181:0/3023599196 shutdown_connections 2024-05-03T20:35:23.825 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:23.823+0000 7fbd11e93700 1 -- 172.21.15.181:0/3023599196 wait complete. 2024-05-03T20:35:23.894 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:35:23.931 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920830 got 141733920836 for osd.4 2024-05-03T20:35:23.931 DEBUG:teuthology.parallel:result is None 2024-05-03T20:35:24.192 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:24 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3023599196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-03T20:35:24.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:24 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3023599196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-03T20:35:24.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:24 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3023599196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-03T20:35:24.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.579+0000 7f6500193700 1 -- 172.21.15.181:0/3710863192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64f8109520 msgr2=0x7f64f8109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:24.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.579+0000 7f6500193700 1 --2- 172.21.15.181:0/3710863192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64f8109520 0x7f64f8109900 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009990 tx=0x7f64f4009ca0 comp rx=0 tx=0).stop 2024-05-03T20:35:24.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.579+0000 7f6500193700 1 -- 172.21.15.181:0/3710863192 shutdown_connections 2024-05-03T20:35:24.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.579+0000 7f6500193700 1 --2- 172.21.15.181:0/3710863192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64f8102450 0x7f64f8110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:24.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.579+0000 7f6500193700 1 --2- 172.21.15.181:0/3710863192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64f8109520 0x7f64f8109900 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:24.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.579+0000 7f6500193700 1 --2- 172.21.15.181:0/3710863192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64f8101a90 0x7f64f8101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:24.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.579+0000 7f6500193700 1 -- 172.21.15.181:0/3710863192 >> 172.21.15.181:0/3710863192 conn(0x7f64f8078a10 msgr2=0x7f64f80ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:24.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.579+0000 7f6500193700 1 -- 172.21.15.181:0/3710863192 shutdown_connections 2024-05-03T20:35:24.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.579+0000 7f6500193700 1 -- 172.21.15.181:0/3710863192 wait complete. 2024-05-03T20:35:24.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.580+0000 7f6500193700 1 Processor -- start 2024-05-03T20:35:24.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.580+0000 7f6500193700 1 -- start start 2024-05-03T20:35:24.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.580+0000 7f6500193700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64f8101a90 0x7f64f819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:24.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.580+0000 7f6500193700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64f8102450 0x7f64f819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:24.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.580+0000 7f6500193700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64f8109520 0x7f64f81999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:24.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.580+0000 7f6500193700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f64f81136b0 con 0x7f64f8102450 2024-05-03T20:35:24.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.580+0000 7f6500193700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f64f8113530 con 0x7f64f8109520 2024-05-03T20:35:24.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.580+0000 7f6500193700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f64f8113830 con 0x7f64f8101a90 2024-05-03T20:35:24.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.581+0000 7f64fd72e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64f8102450 0x7f64f819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:24.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.581+0000 7f64fd72e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64f8102450 0x7f64f819fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50640/0 (socket says 172.21.15.181:50640) 2024-05-03T20:35:24.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.581+0000 7f64fd72e700 1 -- 172.21.15.181:0/3320081622 learned_addr learned my addr 172.21.15.181:0/3320081622 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:24.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.581+0000 7f64fdf2f700 1 --2- 172.21.15.181:0/3320081622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64f8101a90 0x7f64f819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:24.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.581+0000 7f64fe730700 1 --2- 172.21.15.181:0/3320081622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64f8109520 0x7f64f81999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:24.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.581+0000 7f64fd72e700 1 -- 172.21.15.181:0/3320081622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64f8101a90 msgr2=0x7f64f819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:24.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.581+0000 7f64fd72e700 1 --2- 172.21.15.181:0/3320081622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64f8101a90 0x7f64f819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:24.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.581+0000 7f64fd72e700 1 -- 172.21.15.181:0/3320081622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64f8109520 msgr2=0x7f64f81999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:24.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.581+0000 7f64fd72e700 1 --2- 172.21.15.181:0/3320081622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64f8109520 0x7f64f81999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:24.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.581+0000 7f64fd72e700 1 -- 172.21.15.181:0/3320081622 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f819a130 con 0x7f64f8102450 2024-05-03T20:35:24.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.581+0000 7f64fe730700 1 --2- 172.21.15.181:0/3320081622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64f8109520 0x7f64f81999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:35:24.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.581+0000 7f64fdf2f700 1 --2- 172.21.15.181:0/3320081622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64f8101a90 0x7f64f819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:35:24.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.581+0000 7f64fd72e700 1 --2- 172.21.15.181:0/3320081622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64f8102450 0x7f64f819fd00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f64e800b530 tx=0x7f64e800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:24.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.581+0000 7f64eeffd700 1 -- 172.21.15.181:0/3320081622 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e8015020 con 0x7f64f8102450 2024-05-03T20:35:24.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.581+0000 7f6500193700 1 -- 172.21.15.181:0/3320081622 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f819a420 con 0x7f64f8102450 2024-05-03T20:35:24.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.581+0000 7f64eeffd700 1 -- 172.21.15.181:0/3320081622 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64e8011ce0 con 0x7f64f8102450 2024-05-03T20:35:24.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.582+0000 7f64eeffd700 1 -- 172.21.15.181:0/3320081622 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e8010d20 con 0x7f64f8102450 2024-05-03T20:35:24.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.582+0000 7f6500193700 1 -- 172.21.15.181:0/3320081622 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f8102ba0 con 0x7f64f8102450 2024-05-03T20:35:24.584 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.583+0000 7f6500193700 1 -- 172.21.15.181:0/3320081622 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64f819aeb0 con 0x7f64f8102450 2024-05-03T20:35:24.584 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.584+0000 7f64eeffd700 1 -- 172.21.15.181:0/3320081622 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f64e8018020 con 0x7f64f8102450 2024-05-03T20:35:24.584 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.584+0000 7f64eeffd700 1 --2- 172.21.15.181:0/3320081622 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f64e40b2910 0x7f64e40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:24.584 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.584+0000 7f64fdf2f700 1 --2- 172.21.15.181:0/3320081622 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f64e40b2910 0x7f64e40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:24.584 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.585+0000 7f64eeffd700 1 -- 172.21.15.181:0/3320081622 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f64e80e1ab0 con 0x7f64f8102450 2024-05-03T20:35:24.584 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.585+0000 7f64fdf2f700 1 --2- 172.21.15.181:0/3320081622 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f64e40b2910 0x7f64e40b4dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f64f4000c00 tx=0x7f64f4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:24.588 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.589+0000 7f64eeffd700 1 -- 172.21.15.181:0/3320081622 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f64e801f020 con 0x7f64f8102450 2024-05-03T20:35:24.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.801+0000 7f6500193700 1 -- 172.21.15.181:0/3320081622 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f64f802e510 con 0x7f64f8102450 2024-05-03T20:35:24.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.801+0000 7f64eeffd700 1 -- 172.21.15.181:0/3320081622 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f64e80a9780 con 0x7f64f8102450 2024-05-03T20:35:24.801 INFO:teuthology.orchestra.run.smithi181.stdout:313532612643 2024-05-03T20:35:24.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.804+0000 7f6500193700 1 -- 172.21.15.181:0/3320081622 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f64e40b2910 msgr2=0x7f64e40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:24.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.804+0000 7f6500193700 1 --2- 172.21.15.181:0/3320081622 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f64e40b2910 0x7f64e40b4dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f64f4000c00 tx=0x7f64f4005fb0 comp rx=0 tx=0).stop 2024-05-03T20:35:24.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.804+0000 7f6500193700 1 -- 172.21.15.181:0/3320081622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64f8102450 msgr2=0x7f64f819fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:24.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.804+0000 7f6500193700 1 --2- 172.21.15.181:0/3320081622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64f8102450 0x7f64f819fd00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f64e800b530 tx=0x7f64e800b8f0 comp rx=0 tx=0).stop 2024-05-03T20:35:24.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.804+0000 7f6500193700 1 -- 172.21.15.181:0/3320081622 shutdown_connections 2024-05-03T20:35:24.804 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.804+0000 7f6500193700 1 --2- 172.21.15.181:0/3320081622 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f64e40b2910 0x7f64e40b4dd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:24.804 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.804+0000 7f6500193700 1 --2- 172.21.15.181:0/3320081622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64f8109520 0x7f64f81999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:24.804 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.804+0000 7f6500193700 1 --2- 172.21.15.181:0/3320081622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64f8101a90 0x7f64f819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:24.804 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.804+0000 7f6500193700 1 --2- 172.21.15.181:0/3320081622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64f8102450 0x7f64f819fd00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:24.804 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.804+0000 7f6500193700 1 -- 172.21.15.181:0/3320081622 >> 172.21.15.181:0/3320081622 conn(0x7f64f8078a10 msgr2=0x7f64f81001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:24.804 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.804+0000 7f6500193700 1 -- 172.21.15.181:0/3320081622 shutdown_connections 2024-05-03T20:35:24.804 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:24.804+0000 7f6500193700 1 -- 172.21.15.181:0/3320081622 wait complete. 2024-05-03T20:35:25.227 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561080 got 188978561085 for osd.6 2024-05-03T20:35:25.227 DEBUG:teuthology.parallel:result is None 2024-05-03T20:35:25.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:25 smithi196 ceph-mon[32858]: pgmap v338: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:35:25.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:25 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3320081622' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-03T20:35:25.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:25 smithi204 ceph-mon[33595]: pgmap v338: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:35:25.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:25 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3320081622' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-03T20:35:25.430 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:25 smithi181 ceph-mon[29801]: pgmap v338: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:35:25.430 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:25 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3320081622' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-03T20:35:25.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.749+0000 7efcb4f83700 1 -- 172.21.15.181:0/1373670627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcb0102d60 msgr2=0x7efcb010f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:25.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.749+0000 7efcb4f83700 1 --2- 172.21.15.181:0/1373670627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcb0102d60 0x7efcb010f250 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efca800b0a0 tx=0x7efca800b3b0 comp rx=0 tx=0).stop 2024-05-03T20:35:25.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.749+0000 7efcb4f83700 1 -- 172.21.15.181:0/1373670627 shutdown_connections 2024-05-03T20:35:25.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.749+0000 7efcb4f83700 1 --2- 172.21.15.181:0/1373670627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcb0102d60 0x7efcb010f250 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:25.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.749+0000 7efcb4f83700 1 --2- 172.21.15.181:0/1373670627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcb0108410 0x7efcb01087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:25.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.749+0000 7efcb4f83700 1 --2- 172.21.15.181:0/1373670627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcb01023a0 0x7efcb0102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:25.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.749+0000 7efcb4f83700 1 -- 172.21.15.181:0/1373670627 >> 172.21.15.181:0/1373670627 conn(0x7efcb0079350 msgr2=0x7efcb0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:25.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.749+0000 7efcb4f83700 1 -- 172.21.15.181:0/1373670627 shutdown_connections 2024-05-03T20:35:25.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.749+0000 7efcb4f83700 1 -- 172.21.15.181:0/1373670627 wait complete. 2024-05-03T20:35:25.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.750+0000 7efcb4f83700 1 Processor -- start 2024-05-03T20:35:25.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.750+0000 7efcb4f83700 1 -- start start 2024-05-03T20:35:25.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.750+0000 7efcb4f83700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcb01023a0 0x7efcb019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:25.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.750+0000 7efcb4f83700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcb0102d60 0x7efcb019aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:25.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.750+0000 7efcb4f83700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcb0108410 0x7efcb01a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:25.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.750+0000 7efcb4f83700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efcb0112580 con 0x7efcb0108410 2024-05-03T20:35:25.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.750+0000 7efcb4f83700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efcb0112400 con 0x7efcb0102d60 2024-05-03T20:35:25.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.750+0000 7efcb4f83700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efcb0112700 con 0x7efcb01023a0 2024-05-03T20:35:25.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.751+0000 7efcaed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcb0108410 0x7efcb01a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:25.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.751+0000 7efcaed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcb0108410 0x7efcb01a59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57436/0 (socket says 172.21.15.181:57436) 2024-05-03T20:35:25.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.751+0000 7efcaed9d700 1 -- 172.21.15.181:0/842951257 learned_addr learned my addr 172.21.15.181:0/842951257 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:25.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.751+0000 7efcae59c700 1 --2- 172.21.15.181:0/842951257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcb01023a0 0x7efcb019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:25.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.751+0000 7efcadd9b700 1 --2- 172.21.15.181:0/842951257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcb0102d60 0x7efcb019aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:25.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.751+0000 7efcaed9d700 1 -- 172.21.15.181:0/842951257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcb01023a0 msgr2=0x7efcb019a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:25.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.751+0000 7efcaed9d700 1 --2- 172.21.15.181:0/842951257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcb01023a0 0x7efcb019a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:25.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.751+0000 7efcaed9d700 1 -- 172.21.15.181:0/842951257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcb0102d60 msgr2=0x7efcb019aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:25.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.751+0000 7efcaed9d700 1 --2- 172.21.15.181:0/842951257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcb0102d60 0x7efcb019aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:25.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.751+0000 7efcaed9d700 1 -- 172.21.15.181:0/842951257 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb01a5f20 con 0x7efcb0108410 2024-05-03T20:35:25.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.751+0000 7efcaed9d700 1 --2- 172.21.15.181:0/842951257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcb0108410 0x7efcb01a59e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efca800b070 tx=0x7efca800ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:25.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.751+0000 7efc9f7fe700 1 -- 172.21.15.181:0/842951257 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8021070 con 0x7efcb0108410 2024-05-03T20:35:25.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.751+0000 7efc9f7fe700 1 -- 172.21.15.181:0/842951257 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efca8007c80 con 0x7efcb0108410 2024-05-03T20:35:25.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.751+0000 7efc9f7fe700 1 -- 172.21.15.181:0/842951257 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8005440 con 0x7efcb0108410 2024-05-03T20:35:25.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.751+0000 7efcb4f83700 1 -- 172.21.15.181:0/842951257 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb01a60f0 con 0x7efcb0108410 2024-05-03T20:35:25.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.751+0000 7efcb4f83700 1 -- 172.21.15.181:0/842951257 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb01a64f0 con 0x7efcb0108410 2024-05-03T20:35:25.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.752+0000 7efcb4f83700 1 -- 172.21.15.181:0/842951257 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcb006a190 con 0x7efcb0108410 2024-05-03T20:35:25.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.753+0000 7efc9f7fe700 1 -- 172.21.15.181:0/842951257 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7efca8024070 con 0x7efcb0108410 2024-05-03T20:35:25.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.754+0000 7efc9f7fe700 1 --2- 172.21.15.181:0/842951257 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7efc980b2910 0x7efc980b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:25.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.754+0000 7efcae59c700 1 --2- 172.21.15.181:0/842951257 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7efc980b2910 0x7efc980b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:25.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.754+0000 7efc9f7fe700 1 -- 172.21.15.181:0/842951257 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7efca80e3580 con 0x7efcb0108410 2024-05-03T20:35:25.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.755+0000 7efcae59c700 1 --2- 172.21.15.181:0/842951257 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7efc980b2910 0x7efc980b4dd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efca400abd0 tx=0x7efca400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:25.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.759+0000 7efc9f7fe700 1 -- 172.21.15.181:0/842951257 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7efca80ab1d0 con 0x7efcb0108410 2024-05-03T20:35:25.968 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.968+0000 7efcb4f83700 1 -- 172.21.15.181:0/842951257 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7efcb002e6d0 con 0x7efcb0108410 2024-05-03T20:35:25.968 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.969+0000 7efc9f7fe700 1 -- 172.21.15.181:0/842951257 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efca80b31c0 con 0x7efcb0108410 2024-05-03T20:35:25.968 INFO:teuthology.orchestra.run.smithi181.stdout:55834574934 2024-05-03T20:35:25.970 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.971+0000 7efcb4f83700 1 -- 172.21.15.181:0/842951257 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7efc980b2910 msgr2=0x7efc980b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:25.971 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.971+0000 7efcb4f83700 1 --2- 172.21.15.181:0/842951257 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7efc980b2910 0x7efc980b4dd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efca400abd0 tx=0x7efca400a380 comp rx=0 tx=0).stop 2024-05-03T20:35:25.971 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.971+0000 7efcb4f83700 1 -- 172.21.15.181:0/842951257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcb0108410 msgr2=0x7efcb01a59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:25.971 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.971+0000 7efcb4f83700 1 --2- 172.21.15.181:0/842951257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcb0108410 0x7efcb01a59e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efca800b070 tx=0x7efca800ce50 comp rx=0 tx=0).stop 2024-05-03T20:35:25.971 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.971+0000 7efcb4f83700 1 -- 172.21.15.181:0/842951257 shutdown_connections 2024-05-03T20:35:25.971 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.971+0000 7efcb4f83700 1 --2- 172.21.15.181:0/842951257 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7efc980b2910 0x7efc980b4dd0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:25.971 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.971+0000 7efcb4f83700 1 --2- 172.21.15.181:0/842951257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcb0108410 0x7efcb01a59e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:25.971 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.971+0000 7efcb4f83700 1 --2- 172.21.15.181:0/842951257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcb01023a0 0x7efcb019a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:25.971 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.971+0000 7efcb4f83700 1 --2- 172.21.15.181:0/842951257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcb0102d60 0x7efcb019aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:25.971 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.971+0000 7efcb4f83700 1 -- 172.21.15.181:0/842951257 >> 172.21.15.181:0/842951257 conn(0x7efcb0079350 msgr2=0x7efcb0100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:25.971 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.971+0000 7efcb4f83700 1 -- 172.21.15.181:0/842951257 shutdown_connections 2024-05-03T20:35:25.971 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:25.971+0000 7efcb4f83700 1 -- 172.21.15.181:0/842951257 wait complete. 2024-05-03T20:35:26.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:26 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/842951257' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-03T20:35:26.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:26 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/842951257' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-03T20:35:26.369 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:26 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/842951257' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-03T20:35:26.793 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574929 got 55834574934 for osd.1 2024-05-03T20:35:26.794 DEBUG:teuthology.parallel:result is None 2024-05-03T20:35:27.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:27 smithi196 ceph-mon[32858]: pgmap v339: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:35:27.358 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:27 smithi181 ceph-mon[29801]: pgmap v339: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:35:27.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:27 smithi204 ceph-mon[33595]: pgmap v339: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:35:27.621 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612637 got 313532612643 for osd.11 2024-05-03T20:35:27.622 DEBUG:teuthology.parallel:result is None 2024-05-03T20:35:28.777 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.539+0000 7fe7a6a15700 1 -- 172.21.15.181:0/2903607681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7a0104160 msgr2=0x7fe7a0108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:28.777 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.539+0000 7fe7a6a15700 1 --2- 172.21.15.181:0/2903607681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7a0104160 0x7fe7a0108150 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe7940099c0 tx=0x7fe794009cd0 comp rx=0 tx=0).stop 2024-05-03T20:35:28.777 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.540+0000 7fe7a6a15700 1 -- 172.21.15.181:0/2903607681 shutdown_connections 2024-05-03T20:35:28.777 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.540+0000 7fe7a6a15700 1 --2- 172.21.15.181:0/2903607681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7a0108880 0x7fe7a0110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:28.777 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.540+0000 7fe7a6a15700 1 --2- 172.21.15.181:0/2903607681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7a01037b0 0x7fe7a0103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:28.777 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.540+0000 7fe7a6a15700 1 --2- 172.21.15.181:0/2903607681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7a0104160 0x7fe7a0108150 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:28.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.540+0000 7fe7a6a15700 1 -- 172.21.15.181:0/2903607681 >> 172.21.15.181:0/2903607681 conn(0x7fe7a007a580 msgr2=0x7fe7a007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:28.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.540+0000 7fe7a6a15700 1 -- 172.21.15.181:0/2903607681 shutdown_connections 2024-05-03T20:35:28.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.540+0000 7fe7a6a15700 1 -- 172.21.15.181:0/2903607681 wait complete. 2024-05-03T20:35:28.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.540+0000 7fe7a6a15700 1 Processor -- start 2024-05-03T20:35:28.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.540+0000 7fe7a6a15700 1 -- start start 2024-05-03T20:35:28.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.541+0000 7fe7a6a15700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7a01037b0 0x7fe7a019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:28.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.541+0000 7fe7a6a15700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7a0104160 0x7fe7a019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:28.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.541+0000 7fe7a6a15700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7a0108880 0x7fe7a0199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:28.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.541+0000 7fe7a6a15700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a0113640 con 0x7fe7a0104160 2024-05-03T20:35:28.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.541+0000 7fe7a6a15700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a01134c0 con 0x7fe7a0108880 2024-05-03T20:35:28.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.541+0000 7fe7a6a15700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a01137c0 con 0x7fe7a01037b0 2024-05-03T20:35:28.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.541+0000 7fe79f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7a0104160 0x7fe7a019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:28.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.541+0000 7fe79f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7a0104160 0x7fe7a019fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57446/0 (socket says 172.21.15.181:57446) 2024-05-03T20:35:28.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.541+0000 7fe79f7fe700 1 -- 172.21.15.181:0/2234974295 learned_addr learned my addr 172.21.15.181:0/2234974295 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:28.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.541+0000 7fe7a4fb2700 1 --2- 172.21.15.181:0/2234974295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7a0108880 0x7fe7a0199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:28.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.541+0000 7fe79ffff700 1 --2- 172.21.15.181:0/2234974295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7a01037b0 0x7fe7a019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:28.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.541+0000 7fe79f7fe700 1 -- 172.21.15.181:0/2234974295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7a01037b0 msgr2=0x7fe7a019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:28.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.541+0000 7fe79f7fe700 1 --2- 172.21.15.181:0/2234974295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7a01037b0 0x7fe7a019f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:28.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.541+0000 7fe79f7fe700 1 -- 172.21.15.181:0/2234974295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7a0108880 msgr2=0x7fe7a0199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:28.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.541+0000 7fe79f7fe700 1 --2- 172.21.15.181:0/2234974295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7a0108880 0x7fe7a0199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:28.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.541+0000 7fe79f7fe700 1 -- 172.21.15.181:0/2234974295 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7a019a060 con 0x7fe7a0104160 2024-05-03T20:35:28.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.541+0000 7fe7a4fb2700 1 --2- 172.21.15.181:0/2234974295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7a0108880 0x7fe7a0199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:35:28.779 INFO:teuthology.orchestra.run.smithi181.stdout:210453397560 2024-05-03T20:35:28.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.541+0000 7fe79f7fe700 1 --2- 172.21.15.181:0/2234974295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7a0104160 0x7fe7a019fc80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe794006010 tx=0x7fe794004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:28.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.541+0000 7fe79d7fa700 1 -- 172.21.15.181:0/2234974295 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794026070 con 0x7fe7a0104160 2024-05-03T20:35:28.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.541+0000 7fe7a6a15700 1 -- 172.21.15.181:0/2234974295 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7a019a2f0 con 0x7fe7a0104160 2024-05-03T20:35:28.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.541+0000 7fe79d7fa700 1 -- 172.21.15.181:0/2234974295 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe79400f460 con 0x7fe7a0104160 2024-05-03T20:35:28.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.541+0000 7fe79d7fa700 1 -- 172.21.15.181:0/2234974295 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794009e60 con 0x7fe7a0104160 2024-05-03T20:35:28.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.541+0000 7fe7a6a15700 1 -- 172.21.15.181:0/2234974295 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7a01a6610 con 0x7fe7a0104160 2024-05-03T20:35:28.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.542+0000 7fe7a6a15700 1 -- 172.21.15.181:0/2234974295 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7a0104c10 con 0x7fe7a0104160 2024-05-03T20:35:28.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.544+0000 7fe79d7fa700 1 -- 172.21.15.181:0/2234974295 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fe794029070 con 0x7fe7a0104160 2024-05-03T20:35:28.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.544+0000 7fe79d7fa700 1 --2- 172.21.15.181:0/2234974295 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fe7880b2910 0x7fe7880b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:28.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.544+0000 7fe79ffff700 1 --2- 172.21.15.181:0/2234974295 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fe7880b2910 0x7fe7880b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:28.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.545+0000 7fe79d7fa700 1 -- 172.21.15.181:0/2234974295 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7fe7940e2e80 con 0x7fe7a0104160 2024-05-03T20:35:28.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.545+0000 7fe79ffff700 1 --2- 172.21.15.181:0/2234974295 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fe7880b2910 0x7fe7880b4dd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe790005950 tx=0x7fe7900058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:28.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.548+0000 7fe79d7fa700 1 -- 172.21.15.181:0/2234974295 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe794030080 con 0x7fe7a0104160 2024-05-03T20:35:28.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.769+0000 7fe7a6a15700 1 -- 172.21.15.181:0/2234974295 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fe7a002e510 con 0x7fe7a0104160 2024-05-03T20:35:28.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.770+0000 7fe79d7fa700 1 -- 172.21.15.181:0/2234974295 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe794013070 con 0x7fe7a0104160 2024-05-03T20:35:28.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.772+0000 7fe7a6a15700 1 -- 172.21.15.181:0/2234974295 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fe7880b2910 msgr2=0x7fe7880b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:28.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.772+0000 7fe7a6a15700 1 --2- 172.21.15.181:0/2234974295 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fe7880b2910 0x7fe7880b4dd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe790005950 tx=0x7fe7900058e0 comp rx=0 tx=0).stop 2024-05-03T20:35:28.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.772+0000 7fe7a6a15700 1 -- 172.21.15.181:0/2234974295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7a0104160 msgr2=0x7fe7a019fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:28.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.772+0000 7fe7a6a15700 1 --2- 172.21.15.181:0/2234974295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7a0104160 0x7fe7a019fc80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe794006010 tx=0x7fe794004690 comp rx=0 tx=0).stop 2024-05-03T20:35:28.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.772+0000 7fe7a6a15700 1 -- 172.21.15.181:0/2234974295 shutdown_connections 2024-05-03T20:35:28.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.772+0000 7fe7a6a15700 1 --2- 172.21.15.181:0/2234974295 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fe7880b2910 0x7fe7880b4dd0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:28.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.772+0000 7fe7a6a15700 1 --2- 172.21.15.181:0/2234974295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7a0108880 0x7fe7a0199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:28.782 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.772+0000 7fe7a6a15700 1 --2- 172.21.15.181:0/2234974295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7a01037b0 0x7fe7a019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:28.782 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.772+0000 7fe7a6a15700 1 --2- 172.21.15.181:0/2234974295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7a0104160 0x7fe7a019fc80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:28.782 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.772+0000 7fe7a6a15700 1 -- 172.21.15.181:0/2234974295 >> 172.21.15.181:0/2234974295 conn(0x7fe7a007a580 msgr2=0x7fe7a0101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:28.782 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.772+0000 7fe7a6a15700 1 -- 172.21.15.181:0/2234974295 shutdown_connections 2024-05-03T20:35:28.782 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:28.773+0000 7fe7a6a15700 1 -- 172.21.15.181:0/2234974295 wait complete. 2024-05-03T20:35:29.230 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.192+0000 7fe5a168f700 1 -- 172.21.15.181:0/1233978633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe59c108400 msgr2=0x7fe59c1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:29.230 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.192+0000 7fe5a168f700 1 --2- 172.21.15.181:0/1233978633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe59c108400 0x7fe59c1087e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe58c009990 tx=0x7fe58c009ca0 comp rx=0 tx=0).stop 2024-05-03T20:35:29.230 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.192+0000 7fe5a168f700 1 -- 172.21.15.181:0/1233978633 shutdown_connections 2024-05-03T20:35:29.230 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.192+0000 7fe5a168f700 1 --2- 172.21.15.181:0/1233978633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe59c102d50 0x7fe59c10f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:29.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.192+0000 7fe5a168f700 1 --2- 172.21.15.181:0/1233978633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe59c108400 0x7fe59c1087e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:29.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.192+0000 7fe5a168f700 1 --2- 172.21.15.181:0/1233978633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe59c102390 0x7fe59c102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:29.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.192+0000 7fe5a168f700 1 -- 172.21.15.181:0/1233978633 >> 172.21.15.181:0/1233978633 conn(0x7fe59c079310 msgr2=0x7fe59c079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:29.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.192+0000 7fe5a168f700 1 -- 172.21.15.181:0/1233978633 shutdown_connections 2024-05-03T20:35:29.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.192+0000 7fe5a168f700 1 -- 172.21.15.181:0/1233978633 wait complete. 2024-05-03T20:35:29.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.193+0000 7fe5a168f700 1 Processor -- start 2024-05-03T20:35:29.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.193+0000 7fe5a168f700 1 -- start start 2024-05-03T20:35:29.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.193+0000 7fe5a168f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe59c102390 0x7fe59c19f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:29.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.193+0000 7fe5a168f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe59c102d50 0x7fe59c19fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:29.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.193+0000 7fe5a168f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe59c108400 0x7fe59c199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:29.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.193+0000 7fe5a168f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c1125a0 con 0x7fe59c108400 2024-05-03T20:35:29.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.193+0000 7fe5a168f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c112420 con 0x7fe59c102d50 2024-05-03T20:35:29.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.193+0000 7fe5a168f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c112720 con 0x7fe59c102390 2024-05-03T20:35:29.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.193+0000 7fe59b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe59c108400 0x7fe59c199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:29.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.193+0000 7fe59b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe59c108400 0x7fe59c199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57466/0 (socket says 172.21.15.181:57466) 2024-05-03T20:35:29.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.193+0000 7fe59b7fe700 1 -- 172.21.15.181:0/4016468108 learned_addr learned my addr 172.21.15.181:0/4016468108 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:29.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.194+0000 7fe59affd700 1 --2- 172.21.15.181:0/4016468108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe59c102390 0x7fe59c19f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:29.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.194+0000 7fe59b7fe700 1 -- 172.21.15.181:0/4016468108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe59c102390 msgr2=0x7fe59c19f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:29.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.194+0000 7fe59b7fe700 1 --2- 172.21.15.181:0/4016468108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe59c102390 0x7fe59c19f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:29.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.194+0000 7fe59b7fe700 1 -- 172.21.15.181:0/4016468108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe59c102d50 msgr2=0x7fe59c19fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:29.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.194+0000 7fe59a7fc700 1 --2- 172.21.15.181:0/4016468108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe59c102d50 0x7fe59c19fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:29.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.194+0000 7fe59b7fe700 1 --2- 172.21.15.181:0/4016468108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe59c102d50 0x7fe59c19fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:29.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.194+0000 7fe59b7fe700 1 -- 172.21.15.181:0/4016468108 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe59c19a1e0 con 0x7fe59c108400 2024-05-03T20:35:29.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.194+0000 7fe59b7fe700 1 --2- 172.21.15.181:0/4016468108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe59c108400 0x7fe59c199a80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe594007ee0 tx=0x7fe59400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:29.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.194+0000 7fe583fff700 1 -- 172.21.15.181:0/4016468108 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59400dbd0 con 0x7fe59c108400 2024-05-03T20:35:29.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.194+0000 7fe583fff700 1 -- 172.21.15.181:0/4016468108 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe59400dd70 con 0x7fe59c108400 2024-05-03T20:35:29.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.194+0000 7fe5a168f700 1 -- 172.21.15.181:0/4016468108 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe59c19a4d0 con 0x7fe59c108400 2024-05-03T20:35:29.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.194+0000 7fe583fff700 1 -- 172.21.15.181:0/4016468108 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe594005990 con 0x7fe59c108400 2024-05-03T20:35:29.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.194+0000 7fe5a168f700 1 -- 172.21.15.181:0/4016468108 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe59c1a6790 con 0x7fe59c108400 2024-05-03T20:35:29.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.196+0000 7fe5a168f700 1 -- 172.21.15.181:0/4016468108 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe59c06a190 con 0x7fe59c108400 2024-05-03T20:35:29.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.197+0000 7fe583fff700 1 -- 172.21.15.181:0/4016468108 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fe59401a020 con 0x7fe59c108400 2024-05-03T20:35:29.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.197+0000 7fe583fff700 1 --2- 172.21.15.181:0/4016468108 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fe5840b2910 0x7fe5840b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:29.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.197+0000 7fe59affd700 1 --2- 172.21.15.181:0/4016468108 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fe5840b2910 0x7fe5840b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:29.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.197+0000 7fe583fff700 1 -- 172.21.15.181:0/4016468108 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7fe5940e1c00 con 0x7fe59c108400 2024-05-03T20:35:29.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.198+0000 7fe59affd700 1 --2- 172.21.15.181:0/4016468108 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fe5840b2910 0x7fe5840b4dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe58c006010 tx=0x7fe58c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:29.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.202+0000 7fe583fff700 1 -- 172.21.15.181:0/4016468108 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe594010070 con 0x7fe59c108400 2024-05-03T20:35:29.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:29 smithi196 ceph-mon[32858]: pgmap v340: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:35:29.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:29 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/2234974295' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-03T20:35:29.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:29 smithi204 ceph-mon[33595]: pgmap v340: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:35:29.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:29 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/2234974295' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-03T20:35:29.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.428+0000 7fe5a168f700 1 -- 172.21.15.181:0/4016468108 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fe59c1a6960 con 0x7fe59c108400 2024-05-03T20:35:29.428 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:29 smithi181 ceph-mon[29801]: pgmap v340: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:35:29.428 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:29 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/2234974295' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-03T20:35:29.429 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.429+0000 7fe583fff700 1 -- 172.21.15.181:0/4016468108 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe5940a97c0 con 0x7fe59c108400 2024-05-03T20:35:29.429 INFO:teuthology.orchestra.run.smithi181.stdout:34359738459 2024-05-03T20:35:29.431 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.431+0000 7fe5a168f700 1 -- 172.21.15.181:0/4016468108 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fe5840b2910 msgr2=0x7fe5840b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:29.431 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.431+0000 7fe5a168f700 1 --2- 172.21.15.181:0/4016468108 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fe5840b2910 0x7fe5840b4dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe58c006010 tx=0x7fe58c00b540 comp rx=0 tx=0).stop 2024-05-03T20:35:29.431 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.431+0000 7fe5a168f700 1 -- 172.21.15.181:0/4016468108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe59c108400 msgr2=0x7fe59c199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:29.431 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.431+0000 7fe5a168f700 1 --2- 172.21.15.181:0/4016468108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe59c108400 0x7fe59c199a80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe594007ee0 tx=0x7fe59400d3b0 comp rx=0 tx=0).stop 2024-05-03T20:35:29.431 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.431+0000 7fe5a168f700 1 -- 172.21.15.181:0/4016468108 shutdown_connections 2024-05-03T20:35:29.431 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.431+0000 7fe5a168f700 1 --2- 172.21.15.181:0/4016468108 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fe5840b2910 0x7fe5840b4dd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:29.432 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.431+0000 7fe5a168f700 1 --2- 172.21.15.181:0/4016468108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe59c108400 0x7fe59c199a80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:29.432 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.431+0000 7fe5a168f700 1 --2- 172.21.15.181:0/4016468108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe59c102390 0x7fe59c19f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:29.432 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.432+0000 7fe5a168f700 1 --2- 172.21.15.181:0/4016468108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe59c102d50 0x7fe59c19fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:29.432 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.432+0000 7fe5a168f700 1 -- 172.21.15.181:0/4016468108 >> 172.21.15.181:0/4016468108 conn(0x7fe59c079310 msgr2=0x7fe59c101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:29.432 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.432+0000 7fe5a168f700 1 -- 172.21.15.181:0/4016468108 shutdown_connections 2024-05-03T20:35:29.432 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:29.432+0000 7fe5a168f700 1 -- 172.21.15.181:0/4016468108 wait complete. 2024-05-03T20:35:29.758 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397554 got 210453397560 for osd.7 2024-05-03T20:35:29.759 DEBUG:teuthology.parallel:result is None 2024-05-03T20:35:30.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:30 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/4016468108' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-03T20:35:30.454 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:30 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/4016468108' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-03T20:35:30.482 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738454 got 34359738459 for osd.0 2024-05-03T20:35:30.483 DEBUG:teuthology.parallel:result is None 2024-05-03T20:35:30.483 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-03T20:35:30.483 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-03T20:35:30.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:30 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/4016468108' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-03T20:35:30.725 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:35:31.728 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:31 smithi181 ceph-mon[29801]: pgmap v341: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:35:31.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:31 smithi196 ceph-mon[32858]: pgmap v341: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:35:31.881 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:31 smithi204 ceph-mon[33595]: pgmap v341: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:35:32.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.290+0000 7f5823a1a700 1 -- 172.21.15.181:0/3606066350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f581c103750 msgr2=0x7f581c103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:32.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.290+0000 7f5823a1a700 1 --2- 172.21.15.181:0/3606066350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f581c103750 0x7f581c103b30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f580c0099c0 tx=0x7f580c009cd0 comp rx=0 tx=0).stop 2024-05-03T20:35:32.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.291+0000 7f5823a1a700 1 -- 172.21.15.181:0/3606066350 shutdown_connections 2024-05-03T20:35:32.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.291+0000 7f5823a1a700 1 --2- 172.21.15.181:0/3606066350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f581c108850 0x7f581c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:32.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.291+0000 7f5823a1a700 1 --2- 172.21.15.181:0/3606066350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f581c103750 0x7f581c103b30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:32.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.291+0000 7f5823a1a700 1 --2- 172.21.15.181:0/3606066350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f581c104100 0x7f581c108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:32.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.291+0000 7f5823a1a700 1 -- 172.21.15.181:0/3606066350 >> 172.21.15.181:0/3606066350 conn(0x7f581c07a600 msgr2=0x7f581c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:32.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.291+0000 7f5823a1a700 1 -- 172.21.15.181:0/3606066350 shutdown_connections 2024-05-03T20:35:32.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.291+0000 7f5823a1a700 1 -- 172.21.15.181:0/3606066350 wait complete. 2024-05-03T20:35:32.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.292+0000 7f5823a1a700 1 Processor -- start 2024-05-03T20:35:32.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.292+0000 7f5823a1a700 1 -- start start 2024-05-03T20:35:32.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.292+0000 7f5823a1a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f581c103750 0x7f581c19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:32.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.292+0000 7f5823a1a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f581c104100 0x7f581c19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:32.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.292+0000 7f5823a1a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f581c108850 0x7f581c199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:32.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.292+0000 7f5823a1a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f581c1136b0 con 0x7f581c103750 2024-05-03T20:35:32.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.292+0000 7f5823a1a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f581c113530 con 0x7f581c104100 2024-05-03T20:35:32.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.292+0000 7f5823a1a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f581c113830 con 0x7f581c108850 2024-05-03T20:35:32.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.292+0000 7f58217b6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f581c103750 0x7f581c19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:32.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.292+0000 7f58217b6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f581c103750 0x7f581c19f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57484/0 (socket says 172.21.15.181:57484) 2024-05-03T20:35:32.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.292+0000 7f58217b6700 1 -- 172.21.15.181:0/340875034 learned_addr learned my addr 172.21.15.181:0/340875034 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:32.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.292+0000 7f5820fb5700 1 --2- 172.21.15.181:0/340875034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f581c104100 0x7f581c19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:32.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.292+0000 7f5821fb7700 1 --2- 172.21.15.181:0/340875034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f581c108850 0x7f581c199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:32.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.292+0000 7f58217b6700 1 -- 172.21.15.181:0/340875034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f581c108850 msgr2=0x7f581c199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:32.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.292+0000 7f58217b6700 1 --2- 172.21.15.181:0/340875034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f581c108850 0x7f581c199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:32.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.292+0000 7f58217b6700 1 -- 172.21.15.181:0/340875034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f581c104100 msgr2=0x7f581c19fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:32.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.292+0000 7f58217b6700 1 --2- 172.21.15.181:0/340875034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f581c104100 0x7f581c19fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:32.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.292+0000 7f58217b6700 1 -- 172.21.15.181:0/340875034 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f581c19a0f0 con 0x7f581c103750 2024-05-03T20:35:32.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.292+0000 7f5820fb5700 1 --2- 172.21.15.181:0/340875034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f581c104100 0x7f581c19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:35:32.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.293+0000 7f58217b6700 1 --2- 172.21.15.181:0/340875034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f581c103750 0x7f581c19f7d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f580c00b580 tx=0x7f580c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:32.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.293+0000 7f58127fc700 1 -- 172.21.15.181:0/340875034 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c027070 con 0x7f581c103750 2024-05-03T20:35:32.295 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.293+0000 7f58127fc700 1 -- 172.21.15.181:0/340875034 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f580c004020 con 0x7f581c103750 2024-05-03T20:35:32.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.293+0000 7f5823a1a700 1 -- 172.21.15.181:0/340875034 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f581c19a380 con 0x7f581c103750 2024-05-03T20:35:32.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.293+0000 7f58127fc700 1 -- 172.21.15.181:0/340875034 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c022e30 con 0x7f581c103750 2024-05-03T20:35:32.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.293+0000 7f5823a1a700 1 -- 172.21.15.181:0/340875034 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f581c1a66a0 con 0x7f581c103750 2024-05-03T20:35:32.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.294+0000 7f5823a1a700 1 -- 172.21.15.181:0/340875034 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f581c06a190 con 0x7f581c103750 2024-05-03T20:35:32.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.295+0000 7f58127fc700 1 -- 172.21.15.181:0/340875034 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f580c022590 con 0x7f581c103750 2024-05-03T20:35:32.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.296+0000 7f58127fc700 1 --2- 172.21.15.181:0/340875034 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f58080b2910 0x7f58080b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:32.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.296+0000 7f5820fb5700 1 --2- 172.21.15.181:0/340875034 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f58080b2910 0x7f58080b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:32.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.296+0000 7f58127fc700 1 -- 172.21.15.181:0/340875034 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f580c0e34f0 con 0x7f581c103750 2024-05-03T20:35:32.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.296+0000 7f5820fb5700 1 --2- 172.21.15.181:0/340875034 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f58080b2910 0x7f58080b4dd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f581c1030e0 tx=0x7f5818008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:32.299 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.299+0000 7f58127fc700 1 -- 172.21.15.181:0/340875034 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f580c030070 con 0x7f581c103750 2024-05-03T20:35:32.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.497+0000 7f5823a1a700 1 -- 172.21.15.181:0/340875034 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f581c0638e0 con 0x7f58080b2910 2024-05-03T20:35:32.504 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.505+0000 7f58127fc700 1 -- 172.21.15.181:0/340875034 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258767 (secure 0 0 0) 0x7f581c0638e0 con 0x7f58080b2910 2024-05-03T20:35:32.508 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:35:32.509 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.508+0000 7f5823a1a700 1 -- 172.21.15.181:0/340875034 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f58080b2910 msgr2=0x7f58080b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:32.510 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.508+0000 7f5823a1a700 1 --2- 172.21.15.181:0/340875034 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f58080b2910 0x7f58080b4dd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f581c1030e0 tx=0x7f5818008040 comp rx=0 tx=0).stop 2024-05-03T20:35:32.510 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.508+0000 7f5823a1a700 1 -- 172.21.15.181:0/340875034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f581c103750 msgr2=0x7f581c19f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:32.510 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.508+0000 7f5823a1a700 1 --2- 172.21.15.181:0/340875034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f581c103750 0x7f581c19f7d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f580c00b580 tx=0x7f580c0047b0 comp rx=0 tx=0).stop 2024-05-03T20:35:32.510 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.508+0000 7f5823a1a700 1 -- 172.21.15.181:0/340875034 shutdown_connections 2024-05-03T20:35:32.510 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.508+0000 7f5823a1a700 1 --2- 172.21.15.181:0/340875034 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f58080b2910 0x7f58080b4dd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:32.510 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.508+0000 7f5823a1a700 1 --2- 172.21.15.181:0/340875034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f581c108850 0x7f581c199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:32.510 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.508+0000 7f5823a1a700 1 --2- 172.21.15.181:0/340875034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f581c103750 0x7f581c19f7d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:32.510 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.508+0000 7f5823a1a700 1 --2- 172.21.15.181:0/340875034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f581c104100 0x7f581c19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:32.510 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.508+0000 7f5823a1a700 1 -- 172.21.15.181:0/340875034 >> 172.21.15.181:0/340875034 conn(0x7f581c07a600 msgr2=0x7f581c101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:32.510 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.508+0000 7f5823a1a700 1 -- 172.21.15.181:0/340875034 shutdown_connections 2024-05-03T20:35:32.510 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:32.508+0000 7f5823a1a700 1 -- 172.21.15.181:0/340875034 wait complete. 2024-05-03T20:35:32.510 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2024-05-03T20:35:33.239 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2024-05-03T20:35:31.878073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651362,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":982,"num_read_kb":605,"num_write":366,"num_write_kb":2280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2953010,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370256,"kb_used_data":12708,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571568,"statfs":{"total":1151940427776,"available":1151561285632,"internally_reserved":0,"allocated":13012992,"data_stored":7198890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":60,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003052"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.596646+0000","last_change":"2024-05-03T20:33:40.713047+0000","last_active":"2024-05-03T20:34:35.596646+0000","last_peered":"2024-05-03T20:34:35.596646+0000","last_clean":"2024-05-03T20:34:35.596646+0000","last_became_active":"2024-05-03T20:33:40.712718+0000","last_became_peered":"2024-05-03T20:33:40.712718+0000","last_unstale":"2024-05-03T20:34:35.596646+0000","last_undegraded":"2024-05-03T20:34:35.596646+0000","last_fullsized":"2024-05-03T20:34:35.596646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:55:24.816247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.594114+0000","last_change":"2024-05-03T20:33:40.716139+0000","last_active":"2024-05-03T20:34:35.594114+0000","last_peered":"2024-05-03T20:34:35.594114+0000","last_clean":"2024-05-03T20:34:35.594114+0000","last_became_active":"2024-05-03T20:33:40.715808+0000","last_became_peered":"2024-05-03T20:33:40.715808+0000","last_unstale":"2024-05-03T20:34:35.594114+0000","last_undegraded":"2024-05-03T20:34:35.594114+0000","last_fullsized":"2024-05-03T20:34:35.594114+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:36:30.410799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.593940+0000","last_change":"2024-05-03T20:33:40.712650+0000","last_active":"2024-05-03T20:34:35.593940+0000","last_peered":"2024-05-03T20:34:35.593940+0000","last_clean":"2024-05-03T20:34:35.593940+0000","last_became_active":"2024-05-03T20:33:40.712285+0000","last_became_peered":"2024-05-03T20:33:40.712285+0000","last_unstale":"2024-05-03T20:34:35.593940+0000","last_undegraded":"2024-05-03T20:34:35.593940+0000","last_fullsized":"2024-05-03T20:34:35.593940+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:10:25.162097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.592424+0000","last_change":"2024-05-03T20:33:40.716602+0000","last_active":"2024-05-03T20:34:35.592424+0000","last_peered":"2024-05-03T20:34:35.592424+0000","last_clean":"2024-05-03T20:34:35.592424+0000","last_became_active":"2024-05-03T20:33:40.716282+0000","last_became_peered":"2024-05-03T20:33:40.716282+0000","last_unstale":"2024-05-03T20:34:35.592424+0000","last_undegraded":"2024-05-03T20:34:35.592424+0000","last_fullsized":"2024-05-03T20:34:35.592424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:15:27.687359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598334+0000","last_change":"2024-05-03T20:33:40.701780+0000","last_active":"2024-05-03T20:34:35.598334+0000","last_peered":"2024-05-03T20:34:35.598334+0000","last_clean":"2024-05-03T20:34:35.598334+0000","last_became_active":"2024-05-03T20:33:40.701447+0000","last_became_peered":"2024-05-03T20:33:40.701447+0000","last_unstale":"2024-05-03T20:34:35.598334+0000","last_undegraded":"2024-05-03T20:34:35.598334+0000","last_fullsized":"2024-05-03T20:34:35.598334+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:23:53.219489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.594068+0000","last_change":"2024-05-03T20:33:40.721249+0000","last_active":"2024-05-03T20:34:35.594068+0000","last_peered":"2024-05-03T20:34:35.594068+0000","last_clean":"2024-05-03T20:34:35.594068+0000","last_became_active":"2024-05-03T20:33:40.720838+0000","last_became_peered":"2024-05-03T20:33:40.720838+0000","last_unstale":"2024-05-03T20:34:35.594068+0000","last_undegraded":"2024-05-03T20:34:35.594068+0000","last_fullsized":"2024-05-03T20:34:35.594068+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:37:12.868435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.595301+0000","last_change":"2024-05-03T20:33:40.708834+0000","last_active":"2024-05-03T20:34:35.595301+0000","last_peered":"2024-05-03T20:34:35.595301+0000","last_clean":"2024-05-03T20:34:35.595301+0000","last_became_active":"2024-05-03T20:33:40.708471+0000","last_became_peered":"2024-05-03T20:33:40.708471+0000","last_unstale":"2024-05-03T20:34:35.595301+0000","last_undegraded":"2024-05-03T20:34:35.595301+0000","last_fullsized":"2024-05-03T20:34:35.595301+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:54:21.297452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.789211+0000","last_change":"2024-05-03T20:33:40.707078+0000","last_active":"2024-05-03T20:34:35.789211+0000","last_peered":"2024-05-03T20:34:35.789211+0000","last_clean":"2024-05-03T20:34:35.789211+0000","last_became_active":"2024-05-03T20:33:40.706724+0000","last_became_peered":"2024-05-03T20:33:40.706724+0000","last_unstale":"2024-05-03T20:34:35.789211+0000","last_undegraded":"2024-05-03T20:34:35.789211+0000","last_fullsized":"2024-05-03T20:34:35.789211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:54:27.397305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.599112+0000","last_change":"2024-05-03T20:33:40.712580+0000","last_active":"2024-05-03T20:34:35.599112+0000","last_peered":"2024-05-03T20:34:35.599112+0000","last_clean":"2024-05-03T20:34:35.599112+0000","last_became_active":"2024-05-03T20:33:40.712252+0000","last_became_peered":"2024-05-03T20:33:40.712252+0000","last_unstale":"2024-05-03T20:34:35.599112+0000","last_undegraded":"2024-05-03T20:34:35.599112+0000","last_fullsized":"2024-05-03T20:34:35.599112+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:59:11.095956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.597225+0000","last_change":"2024-05-03T20:34:35.597225+0000","last_active":"2024-05-03T20:34:35.597225+0000","last_peered":"2024-05-03T20:34:35.597225+0000","last_clean":"2024-05-03T20:34:35.597225+0000","last_became_active":"2024-05-03T20:34:35.596786+0000","last_became_peered":"2024-05-03T20:34:35.596786+0000","last_unstale":"2024-05-03T20:34:35.597225+0000","last_undegraded":"2024-05-03T20:34:35.597225+0000","last_fullsized":"2024-05-03T20:34:35.597225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:04:14.637428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.595967+0000","last_change":"2024-05-03T20:33:40.711383+0000","last_active":"2024-05-03T20:34:35.595967+0000","last_peered":"2024-05-03T20:34:35.595967+0000","last_clean":"2024-05-03T20:34:35.595967+0000","last_became_active":"2024-05-03T20:33:40.710870+0000","last_became_peered":"2024-05-03T20:33:40.710870+0000","last_unstale":"2024-05-03T20:34:35.595967+0000","last_undegraded":"2024-05-03T20:34:35.595967+0000","last_fullsized":"2024-05-03T20:34:35.595967+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:49:18.905954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.592015+0000","last_change":"2024-05-03T20:33:40.713749+0000","last_active":"2024-05-03T20:34:35.592015+0000","last_peered":"2024-05-03T20:34:35.592015+0000","last_clean":"2024-05-03T20:34:35.592015+0000","last_became_active":"2024-05-03T20:33:40.713359+0000","last_became_peered":"2024-05-03T20:33:40.713359+0000","last_unstale":"2024-05-03T20:34:35.592015+0000","last_undegraded":"2024-05-03T20:34:35.592015+0000","last_fullsized":"2024-05-03T20:34:35.592015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:40:46.840267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.931512+0000","last_change":"2024-05-03T20:33:40.708434+0000","last_active":"2024-05-03T20:34:35.931512+0000","last_peered":"2024-05-03T20:34:35.931512+0000","last_clean":"2024-05-03T20:34:35.931512+0000","last_became_active":"2024-05-03T20:33:40.707825+0000","last_became_peered":"2024-05-03T20:33:40.707825+0000","last_unstale":"2024-05-03T20:34:35.931512+0000","last_undegraded":"2024-05-03T20:34:35.931512+0000","last_fullsized":"2024-05-03T20:34:35.931512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:42:49.837828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.593883+0000","last_change":"2024-05-03T20:33:40.714974+0000","last_active":"2024-05-03T20:34:35.593883+0000","last_peered":"2024-05-03T20:34:35.593883+0000","last_clean":"2024-05-03T20:34:35.593883+0000","last_became_active":"2024-05-03T20:33:40.714612+0000","last_became_peered":"2024-05-03T20:33:40.714612+0000","last_unstale":"2024-05-03T20:34:35.593883+0000","last_undegraded":"2024-05-03T20:34:35.593883+0000","last_fullsized":"2024-05-03T20:34:35.593883+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:34:40.766696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.593792+0000","last_change":"2024-05-03T20:33:40.713443+0000","last_active":"2024-05-03T20:34:35.593792+0000","last_peered":"2024-05-03T20:34:35.593792+0000","last_clean":"2024-05-03T20:34:35.593792+0000","last_became_active":"2024-05-03T20:33:40.712803+0000","last_became_peered":"2024-05-03T20:33:40.712803+0000","last_unstale":"2024-05-03T20:34:35.593792+0000","last_undegraded":"2024-05-03T20:34:35.593792+0000","last_fullsized":"2024-05-03T20:34:35.593792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:26:46.942918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.597473+0000","last_change":"2024-05-03T20:33:40.715667+0000","last_active":"2024-05-03T20:34:35.597473+0000","last_peered":"2024-05-03T20:34:35.597473+0000","last_clean":"2024-05-03T20:34:35.597473+0000","last_became_active":"2024-05-03T20:33:40.715291+0000","last_became_peered":"2024-05-03T20:33:40.715291+0000","last_unstale":"2024-05-03T20:34:35.597473+0000","last_undegraded":"2024-05-03T20:34:35.597473+0000","last_fullsized":"2024-05-03T20:34:35.597473+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:21:56.061022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.596902+0000","last_change":"2024-05-03T20:33:40.714495+0000","last_active":"2024-05-03T20:34:35.596902+0000","last_peered":"2024-05-03T20:34:35.596902+0000","last_clean":"2024-05-03T20:34:35.596902+0000","last_became_active":"2024-05-03T20:33:40.714176+0000","last_became_peered":"2024-05-03T20:33:40.714176+0000","last_unstale":"2024-05-03T20:34:35.596902+0000","last_undegraded":"2024-05-03T20:34:35.596902+0000","last_fullsized":"2024-05-03T20:34:35.596902+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:26:34.299414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.591286+0000","last_change":"2024-05-03T20:33:40.705512+0000","last_active":"2024-05-03T20:34:35.591286+0000","last_peered":"2024-05-03T20:34:35.591286+0000","last_clean":"2024-05-03T20:34:35.591286+0000","last_became_active":"2024-05-03T20:33:40.705187+0000","last_became_peered":"2024-05-03T20:33:40.705187+0000","last_unstale":"2024-05-03T20:34:35.591286+0000","last_undegraded":"2024-05-03T20:34:35.591286+0000","last_fullsized":"2024-05-03T20:34:35.591286+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:28:56.412004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.594429+0000","last_change":"2024-05-03T20:33:40.706780+0000","last_active":"2024-05-03T20:34:35.594429+0000","last_peered":"2024-05-03T20:34:35.594429+0000","last_clean":"2024-05-03T20:34:35.594429+0000","last_became_active":"2024-05-03T20:33:40.706439+0000","last_became_peered":"2024-05-03T20:33:40.706439+0000","last_unstale":"2024-05-03T20:34:35.594429+0000","last_undegraded":"2024-05-03T20:34:35.594429+0000","last_fullsized":"2024-05-03T20:34:35.594429+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:43:03.182219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":133,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:35:29.611412+0000","last_change":"2024-05-03T20:33:37.680899+0000","last_active":"2024-05-03T20:35:29.611412+0000","last_peered":"2024-05-03T20:35:29.611412+0000","last_clean":"2024-05-03T20:35:29.611412+0000","last_became_active":"2024-05-03T20:33:37.680602+0000","last_became_peered":"2024-05-03T20:33:37.680602+0000","last_unstale":"2024-05-03T20:35:29.611412+0000","last_undegraded":"2024-05-03T20:35:29.611412+0000","last_fullsized":"2024-05-03T20:35:29.611412+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:39:50.865486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.789426+0000","last_change":"2024-05-03T20:33:40.710869+0000","last_active":"2024-05-03T20:34:35.789426+0000","last_peered":"2024-05-03T20:34:35.789426+0000","last_clean":"2024-05-03T20:34:35.789426+0000","last_became_active":"2024-05-03T20:33:40.710552+0000","last_became_peered":"2024-05-03T20:33:40.710552+0000","last_unstale":"2024-05-03T20:34:35.789426+0000","last_undegraded":"2024-05-03T20:34:35.789426+0000","last_fullsized":"2024-05-03T20:34:35.789426+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:18:59.947702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.789757+0000","last_change":"2024-05-03T20:33:38.237387+0000","last_active":"2024-05-03T20:34:35.789757+0000","last_peered":"2024-05-03T20:34:35.789757+0000","last_clean":"2024-05-03T20:34:35.789757+0000","last_became_active":"2024-05-03T20:33:38.237029+0000","last_became_peered":"2024-05-03T20:33:38.237029+0000","last_unstale":"2024-05-03T20:34:35.789757+0000","last_undegraded":"2024-05-03T20:34:35.789757+0000","last_fullsized":"2024-05-03T20:34:35.789757+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:58:58.360973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.594278+0000","last_change":"2024-05-03T20:33:40.712653+0000","last_active":"2024-05-03T20:34:35.594278+0000","last_peered":"2024-05-03T20:34:35.594278+0000","last_clean":"2024-05-03T20:34:35.594278+0000","last_became_active":"2024-05-03T20:33:40.712283+0000","last_became_peered":"2024-05-03T20:33:40.712283+0000","last_unstale":"2024-05-03T20:34:35.594278+0000","last_undegraded":"2024-05-03T20:34:35.594278+0000","last_fullsized":"2024-05-03T20:34:35.594278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:28:56.388292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.611411+0000","last_change":"2024-05-03T20:34:35.610936+0000","last_active":"2024-05-03T20:34:35.611411+0000","last_peered":"2024-05-03T20:34:35.611411+0000","last_clean":"2024-05-03T20:34:35.611411+0000","last_became_active":"2024-05-03T20:34:35.597835+0000","last_became_peered":"2024-05-03T20:34:35.597835+0000","last_unstale":"2024-05-03T20:34:35.611411+0000","last_undegraded":"2024-05-03T20:34:35.611411+0000","last_fullsized":"2024-05-03T20:34:35.611411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:35:54.327195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598349+0000","last_change":"2024-05-03T20:33:40.711274+0000","last_active":"2024-05-03T20:34:35.598349+0000","last_peered":"2024-05-03T20:34:35.598349+0000","last_clean":"2024-05-03T20:34:35.598349+0000","last_became_active":"2024-05-03T20:33:40.710917+0000","last_became_peered":"2024-05-03T20:33:40.710917+0000","last_unstale":"2024-05-03T20:34:35.598349+0000","last_undegraded":"2024-05-03T20:34:35.598349+0000","last_fullsized":"2024-05-03T20:34:35.598349+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:22:55.969325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598238+0000","last_change":"2024-05-03T20:33:37.681383+0000","last_active":"2024-05-03T20:34:35.598238+0000","last_peered":"2024-05-03T20:34:35.598238+0000","last_clean":"2024-05-03T20:34:35.598238+0000","last_became_active":"2024-05-03T20:33:37.680941+0000","last_became_peered":"2024-05-03T20:33:37.680941+0000","last_unstale":"2024-05-03T20:34:35.598238+0000","last_undegraded":"2024-05-03T20:34:35.598238+0000","last_fullsized":"2024-05-03T20:34:35.598238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:03:26.083579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.599057+0000","last_change":"2024-05-03T20:33:40.711762+0000","last_active":"2024-05-03T20:34:35.599057+0000","last_peered":"2024-05-03T20:34:35.599057+0000","last_clean":"2024-05-03T20:34:35.599057+0000","last_became_active":"2024-05-03T20:33:40.710630+0000","last_became_peered":"2024-05-03T20:33:40.710630+0000","last_unstale":"2024-05-03T20:34:35.599057+0000","last_undegraded":"2024-05-03T20:34:35.599057+0000","last_fullsized":"2024-05-03T20:34:35.599057+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:42:23.728916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":123,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:35:25.607667+0000","last_change":"2024-05-03T20:33:37.685895+0000","last_active":"2024-05-03T20:35:25.607667+0000","last_peered":"2024-05-03T20:35:25.607667+0000","last_clean":"2024-05-03T20:35:25.607667+0000","last_became_active":"2024-05-03T20:33:37.685399+0000","last_became_peered":"2024-05-03T20:33:37.685399+0000","last_unstale":"2024-05-03T20:35:25.607667+0000","last_undegraded":"2024-05-03T20:35:25.607667+0000","last_fullsized":"2024-05-03T20:35:25.607667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:09:29.932599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.597868+0000","last_change":"2024-05-03T20:33:40.704967+0000","last_active":"2024-05-03T20:34:35.597868+0000","last_peered":"2024-05-03T20:34:35.597868+0000","last_clean":"2024-05-03T20:34:35.597868+0000","last_became_active":"2024-05-03T20:33:40.704583+0000","last_became_peered":"2024-05-03T20:33:40.704583+0000","last_unstale":"2024-05-03T20:34:35.597868+0000","last_undegraded":"2024-05-03T20:34:35.597868+0000","last_fullsized":"2024-05-03T20:34:35.597868+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:46:31.720865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.593998+0000","last_change":"2024-05-03T20:33:37.680798+0000","last_active":"2024-05-03T20:34:35.593998+0000","last_peered":"2024-05-03T20:34:35.593998+0000","last_clean":"2024-05-03T20:34:35.593998+0000","last_became_active":"2024-05-03T20:33:37.680344+0000","last_became_peered":"2024-05-03T20:33:37.680344+0000","last_unstale":"2024-05-03T20:34:35.593998+0000","last_undegraded":"2024-05-03T20:34:35.593998+0000","last_fullsized":"2024-05-03T20:34:35.593998+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:26:32.037104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.791470+0000","last_change":"2024-05-03T20:34:35.790972+0000","last_active":"2024-05-03T20:34:35.791470+0000","last_peered":"2024-05-03T20:34:35.791470+0000","last_clean":"2024-05-03T20:34:35.791470+0000","last_became_active":"2024-05-03T20:34:35.790574+0000","last_became_peered":"2024-05-03T20:34:35.790574+0000","last_unstale":"2024-05-03T20:34:35.791470+0000","last_undegraded":"2024-05-03T20:34:35.791470+0000","last_fullsized":"2024-05-03T20:34:35.791470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:19:16.992618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.931345+0000","last_change":"2024-05-03T20:33:37.684230+0000","last_active":"2024-05-03T20:34:35.931345+0000","last_peered":"2024-05-03T20:34:35.931345+0000","last_clean":"2024-05-03T20:34:35.931345+0000","last_became_active":"2024-05-03T20:33:37.683844+0000","last_became_peered":"2024-05-03T20:33:37.683844+0000","last_unstale":"2024-05-03T20:34:35.931345+0000","last_undegraded":"2024-05-03T20:34:35.931345+0000","last_fullsized":"2024-05-03T20:34:35.931345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:46:23.417163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.789235+0000","last_change":"2024-05-03T20:33:40.706554+0000","last_active":"2024-05-03T20:34:35.789235+0000","last_peered":"2024-05-03T20:34:35.789235+0000","last_clean":"2024-05-03T20:34:35.789235+0000","last_became_active":"2024-05-03T20:33:40.706142+0000","last_became_peered":"2024-05-03T20:33:40.706142+0000","last_unstale":"2024-05-03T20:34:35.789235+0000","last_undegraded":"2024-05-03T20:34:35.789235+0000","last_fullsized":"2024-05-03T20:34:35.789235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:01:28.324024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.597925+0000","last_change":"2024-05-03T20:33:37.683497+0000","last_active":"2024-05-03T20:34:35.597925+0000","last_peered":"2024-05-03T20:34:35.597925+0000","last_clean":"2024-05-03T20:34:35.597925+0000","last_became_active":"2024-05-03T20:33:37.683178+0000","last_became_peered":"2024-05-03T20:33:37.683178+0000","last_unstale":"2024-05-03T20:34:35.597925+0000","last_undegraded":"2024-05-03T20:34:35.597925+0000","last_fullsized":"2024-05-03T20:34:35.597925+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:07:29.771786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.590905+0000","last_change":"2024-05-03T20:33:40.705447+0000","last_active":"2024-05-03T20:34:35.590905+0000","last_peered":"2024-05-03T20:34:35.590905+0000","last_clean":"2024-05-03T20:34:35.590905+0000","last_became_active":"2024-05-03T20:33:40.705102+0000","last_became_peered":"2024-05-03T20:33:40.705102+0000","last_unstale":"2024-05-03T20:34:35.590905+0000","last_undegraded":"2024-05-03T20:34:35.590905+0000","last_fullsized":"2024-05-03T20:34:35.590905+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:55:55.354498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":209,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:35:27.609764+0000","last_change":"2024-05-03T20:33:37.681449+0000","last_active":"2024-05-03T20:35:27.609764+0000","last_peered":"2024-05-03T20:35:27.609764+0000","last_clean":"2024-05-03T20:35:27.609764+0000","last_became_active":"2024-05-03T20:33:37.680798+0000","last_became_peered":"2024-05-03T20:33:37.680798+0000","last_unstale":"2024-05-03T20:35:27.609764+0000","last_undegraded":"2024-05-03T20:35:27.609764+0000","last_fullsized":"2024-05-03T20:35:27.609764+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:42:10.029261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":180,"num_read_kb":180,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598976+0000","last_change":"2024-05-03T20:33:40.708872+0000","last_active":"2024-05-03T20:34:35.598976+0000","last_peered":"2024-05-03T20:34:35.598976+0000","last_clean":"2024-05-03T20:34:35.598976+0000","last_became_active":"2024-05-03T20:33:40.708535+0000","last_became_peered":"2024-05-03T20:33:40.708535+0000","last_unstale":"2024-05-03T20:34:35.598976+0000","last_undegraded":"2024-05-03T20:34:35.598976+0000","last_fullsized":"2024-05-03T20:34:35.598976+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:21:08.949065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.938964+0000","last_change":"2024-05-03T20:34:35.938519+0000","last_active":"2024-05-03T20:34:35.938964+0000","last_peered":"2024-05-03T20:34:35.938964+0000","last_clean":"2024-05-03T20:34:35.938964+0000","last_became_active":"2024-05-03T20:34:35.932521+0000","last_became_peered":"2024-05-03T20:34:35.932521+0000","last_unstale":"2024-05-03T20:34:35.938964+0000","last_undegraded":"2024-05-03T20:34:35.938964+0000","last_fullsized":"2024-05-03T20:34:35.938964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:44:28.361138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,6],"acting":[11,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.594170+0000","last_change":"2024-05-03T20:33:40.713991+0000","last_active":"2024-05-03T20:34:35.594170+0000","last_peered":"2024-05-03T20:34:35.594170+0000","last_clean":"2024-05-03T20:34:35.594170+0000","last_became_active":"2024-05-03T20:33:40.713661+0000","last_became_peered":"2024-05-03T20:33:40.713661+0000","last_unstale":"2024-05-03T20:34:35.594170+0000","last_undegraded":"2024-05-03T20:34:35.594170+0000","last_fullsized":"2024-05-03T20:34:35.594170+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:08:20.818863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.599169+0000","last_change":"2024-05-03T20:33:38.237724+0000","last_active":"2024-05-03T20:34:35.599169+0000","last_peered":"2024-05-03T20:34:35.599169+0000","last_clean":"2024-05-03T20:34:35.599169+0000","last_became_active":"2024-05-03T20:33:38.237340+0000","last_became_peered":"2024-05-03T20:33:38.237340+0000","last_unstale":"2024-05-03T20:34:35.599169+0000","last_undegraded":"2024-05-03T20:34:35.599169+0000","last_fullsized":"2024-05-03T20:34:35.599169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:13:35.567527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598543+0000","last_change":"2024-05-03T20:33:40.707454+0000","last_active":"2024-05-03T20:34:35.598543+0000","last_peered":"2024-05-03T20:34:35.598543+0000","last_clean":"2024-05-03T20:34:35.598543+0000","last_became_active":"2024-05-03T20:33:40.707080+0000","last_became_peered":"2024-05-03T20:33:40.707080+0000","last_unstale":"2024-05-03T20:34:35.598543+0000","last_undegraded":"2024-05-03T20:34:35.598543+0000","last_fullsized":"2024-05-03T20:34:35.598543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:12:18.432048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'30","reported_seq":243,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:35:27.609520+0000","last_change":"2024-05-03T20:33:37.681144+0000","last_active":"2024-05-03T20:35:27.609520+0000","last_peered":"2024-05-03T20:35:27.609520+0000","last_clean":"2024-05-03T20:35:27.609520+0000","last_became_active":"2024-05-03T20:33:37.680531+0000","last_became_peered":"2024-05-03T20:33:37.680531+0000","last_unstale":"2024-05-03T20:35:27.609520+0000","last_undegraded":"2024-05-03T20:35:27.609520+0000","last_fullsized":"2024-05-03T20:35:27.609520+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:45:53.536622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53165,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":188,"num_read_kb":145,"num_write":32,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598177+0000","last_change":"2024-05-03T20:33:40.711336+0000","last_active":"2024-05-03T20:34:35.598177+0000","last_peered":"2024-05-03T20:34:35.598177+0000","last_clean":"2024-05-03T20:34:35.598177+0000","last_became_active":"2024-05-03T20:33:40.710991+0000","last_became_peered":"2024-05-03T20:33:40.710991+0000","last_unstale":"2024-05-03T20:34:35.598177+0000","last_undegraded":"2024-05-03T20:34:35.598177+0000","last_fullsized":"2024-05-03T20:34:35.598177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:59:44.729974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.593972+0000","last_change":"2024-05-03T20:33:37.682280+0000","last_active":"2024-05-03T20:34:35.593972+0000","last_peered":"2024-05-03T20:34:35.593972+0000","last_clean":"2024-05-03T20:34:35.593972+0000","last_became_active":"2024-05-03T20:33:37.681612+0000","last_became_peered":"2024-05-03T20:33:37.681612+0000","last_unstale":"2024-05-03T20:34:35.593972+0000","last_undegraded":"2024-05-03T20:34:35.593972+0000","last_fullsized":"2024-05-03T20:34:35.593972+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:50:05.310560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.931335+0000","last_change":"2024-05-03T20:33:40.710309+0000","last_active":"2024-05-03T20:34:35.931335+0000","last_peered":"2024-05-03T20:34:35.931335+0000","last_clean":"2024-05-03T20:34:35.931335+0000","last_became_active":"2024-05-03T20:33:40.710015+0000","last_became_peered":"2024-05-03T20:33:40.710015+0000","last_unstale":"2024-05-03T20:34:35.931335+0000","last_undegraded":"2024-05-03T20:34:35.931335+0000","last_fullsized":"2024-05-03T20:34:35.931335+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:05:04.042635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598619+0000","last_change":"2024-05-03T20:33:37.679525+0000","last_active":"2024-05-03T20:34:35.598619+0000","last_peered":"2024-05-03T20:34:35.598619+0000","last_clean":"2024-05-03T20:34:35.598619+0000","last_became_active":"2024-05-03T20:33:37.678905+0000","last_became_peered":"2024-05-03T20:33:37.678905+0000","last_unstale":"2024-05-03T20:34:35.598619+0000","last_undegraded":"2024-05-03T20:34:35.598619+0000","last_fullsized":"2024-05-03T20:34:35.598619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:17:06.225255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.595860+0000","last_change":"2024-05-03T20:33:40.717613+0000","last_active":"2024-05-03T20:34:35.595860+0000","last_peered":"2024-05-03T20:34:35.595860+0000","last_clean":"2024-05-03T20:34:35.595860+0000","last_became_active":"2024-05-03T20:33:40.717348+0000","last_became_peered":"2024-05-03T20:33:40.717348+0000","last_unstale":"2024-05-03T20:34:35.595860+0000","last_undegraded":"2024-05-03T20:34:35.595860+0000","last_fullsized":"2024-05-03T20:34:35.595860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:20:31.493731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'6","reported_seq":124,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:35:25.606859+0000","last_change":"2024-05-03T20:33:37.683083+0000","last_active":"2024-05-03T20:35:25.606859+0000","last_peered":"2024-05-03T20:35:25.606859+0000","last_clean":"2024-05-03T20:35:25.606859+0000","last_became_active":"2024-05-03T20:33:37.682700+0000","last_became_peered":"2024-05-03T20:33:37.682700+0000","last_unstale":"2024-05-03T20:35:25.606859+0000","last_undegraded":"2024-05-03T20:35:25.606859+0000","last_fullsized":"2024-05-03T20:35:25.606859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:47:18.294314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":93,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.595088+0000","last_change":"2024-05-03T20:33:40.711711+0000","last_active":"2024-05-03T20:34:35.595088+0000","last_peered":"2024-05-03T20:34:35.595088+0000","last_clean":"2024-05-03T20:34:35.595088+0000","last_became_active":"2024-05-03T20:33:40.711116+0000","last_became_peered":"2024-05-03T20:33:40.711116+0000","last_unstale":"2024-05-03T20:34:35.595088+0000","last_undegraded":"2024-05-03T20:34:35.595088+0000","last_fullsized":"2024-05-03T20:34:35.595088+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:53:18.231575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.597738+0000","last_change":"2024-05-03T20:33:38.238005+0000","last_active":"2024-05-03T20:34:35.597738+0000","last_peered":"2024-05-03T20:34:35.597738+0000","last_clean":"2024-05-03T20:34:35.597738+0000","last_became_active":"2024-05-03T20:33:38.237636+0000","last_became_peered":"2024-05-03T20:33:38.237636+0000","last_unstale":"2024-05-03T20:34:35.597738+0000","last_undegraded":"2024-05-03T20:34:35.597738+0000","last_fullsized":"2024-05-03T20:34:35.597738+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:49:15.501715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.599844+0000","last_change":"2024-05-03T20:33:40.716317+0000","last_active":"2024-05-03T20:34:35.599844+0000","last_peered":"2024-05-03T20:34:35.599844+0000","last_clean":"2024-05-03T20:34:35.599844+0000","last_became_active":"2024-05-03T20:33:40.715780+0000","last_became_peered":"2024-05-03T20:33:40.715780+0000","last_unstale":"2024-05-03T20:34:35.599844+0000","last_undegraded":"2024-05-03T20:34:35.599844+0000","last_fullsized":"2024-05-03T20:34:35.599844+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:21:46.186992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598597+0000","last_change":"2024-05-03T20:33:37.679690+0000","last_active":"2024-05-03T20:34:35.598597+0000","last_peered":"2024-05-03T20:34:35.598597+0000","last_clean":"2024-05-03T20:34:35.598597+0000","last_became_active":"2024-05-03T20:33:37.679214+0000","last_became_peered":"2024-05-03T20:33:37.679214+0000","last_unstale":"2024-05-03T20:34:35.598597+0000","last_undegraded":"2024-05-03T20:34:35.598597+0000","last_fullsized":"2024-05-03T20:34:35.598597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:08:54.467850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.595516+0000","last_change":"2024-05-03T20:33:40.714800+0000","last_active":"2024-05-03T20:34:35.595516+0000","last_peered":"2024-05-03T20:34:35.595516+0000","last_clean":"2024-05-03T20:34:35.595516+0000","last_became_active":"2024-05-03T20:33:40.714417+0000","last_became_peered":"2024-05-03T20:33:40.714417+0000","last_unstale":"2024-05-03T20:34:35.595516+0000","last_undegraded":"2024-05-03T20:34:35.595516+0000","last_fullsized":"2024-05-03T20:34:35.595516+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:06:37.110378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.591103+0000","last_change":"2024-05-03T20:33:38.237270+0000","last_active":"2024-05-03T20:34:35.591103+0000","last_peered":"2024-05-03T20:34:35.591103+0000","last_clean":"2024-05-03T20:34:35.591103+0000","last_became_active":"2024-05-03T20:33:38.236917+0000","last_became_peered":"2024-05-03T20:33:38.236917+0000","last_unstale":"2024-05-03T20:34:35.591103+0000","last_undegraded":"2024-05-03T20:34:35.591103+0000","last_fullsized":"2024-05-03T20:34:35.591103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:38:58.079905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.594215+0000","last_change":"2024-05-03T20:33:40.707368+0000","last_active":"2024-05-03T20:34:35.594215+0000","last_peered":"2024-05-03T20:34:35.594215+0000","last_clean":"2024-05-03T20:34:35.594215+0000","last_became_active":"2024-05-03T20:33:40.707049+0000","last_became_peered":"2024-05-03T20:33:40.707049+0000","last_unstale":"2024-05-03T20:34:35.594215+0000","last_undegraded":"2024-05-03T20:34:35.594215+0000","last_fullsized":"2024-05-03T20:34:35.594215+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:02:57.790142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'6","reported_seq":126,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:35:28.610728+0000","last_change":"2024-05-03T20:33:37.679007+0000","last_active":"2024-05-03T20:35:28.610728+0000","last_peered":"2024-05-03T20:35:28.610728+0000","last_clean":"2024-05-03T20:35:28.610728+0000","last_became_active":"2024-05-03T20:33:37.678609+0000","last_became_peered":"2024-05-03T20:33:37.678609+0000","last_unstale":"2024-05-03T20:35:28.610728+0000","last_undegraded":"2024-05-03T20:35:28.610728+0000","last_fullsized":"2024-05-03T20:35:28.610728+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:30:17.622905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.594121+0000","last_change":"2024-05-03T20:33:40.713820+0000","last_active":"2024-05-03T20:34:35.594121+0000","last_peered":"2024-05-03T20:34:35.594121+0000","last_clean":"2024-05-03T20:34:35.594121+0000","last_became_active":"2024-05-03T20:33:40.713487+0000","last_became_peered":"2024-05-03T20:33:40.713487+0000","last_unstale":"2024-05-03T20:34:35.594121+0000","last_undegraded":"2024-05-03T20:34:35.594121+0000","last_fullsized":"2024-05-03T20:34:35.594121+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:23:03.724180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598253+0000","last_change":"2024-05-03T20:33:40.716663+0000","last_active":"2024-05-03T20:34:35.598253+0000","last_peered":"2024-05-03T20:34:35.598253+0000","last_clean":"2024-05-03T20:34:35.598253+0000","last_became_active":"2024-05-03T20:33:40.716201+0000","last_became_peered":"2024-05-03T20:33:40.716201+0000","last_unstale":"2024-05-03T20:34:35.598253+0000","last_undegraded":"2024-05-03T20:34:35.598253+0000","last_fullsized":"2024-05-03T20:34:35.598253+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:38:12.975351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":125,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:35:30.613068+0000","last_change":"2024-05-03T20:33:38.237285+0000","last_active":"2024-05-03T20:35:30.613068+0000","last_peered":"2024-05-03T20:35:30.613068+0000","last_clean":"2024-05-03T20:35:30.613068+0000","last_became_active":"2024-05-03T20:33:38.236825+0000","last_became_peered":"2024-05-03T20:33:38.236825+0000","last_unstale":"2024-05-03T20:35:30.613068+0000","last_undegraded":"2024-05-03T20:35:30.613068+0000","last_fullsized":"2024-05-03T20:35:30.613068+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:46:52.963953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.789739+0000","last_change":"2024-05-03T20:33:40.713271+0000","last_active":"2024-05-03T20:34:35.789739+0000","last_peered":"2024-05-03T20:34:35.789739+0000","last_clean":"2024-05-03T20:34:35.789739+0000","last_became_active":"2024-05-03T20:33:40.712972+0000","last_became_peered":"2024-05-03T20:33:40.712972+0000","last_unstale":"2024-05-03T20:34:35.789739+0000","last_undegraded":"2024-05-03T20:34:35.789739+0000","last_fullsized":"2024-05-03T20:34:35.789739+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:39:46.762096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.591878+0000","last_change":"2024-05-03T20:33:37.682196+0000","last_active":"2024-05-03T20:34:35.591878+0000","last_peered":"2024-05-03T20:34:35.591878+0000","last_clean":"2024-05-03T20:34:35.591878+0000","last_became_active":"2024-05-03T20:33:37.681671+0000","last_became_peered":"2024-05-03T20:33:37.681671+0000","last_unstale":"2024-05-03T20:34:35.591878+0000","last_undegraded":"2024-05-03T20:34:35.591878+0000","last_fullsized":"2024-05-03T20:34:35.591878+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:53:26.866288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":102,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.595294+0000","last_change":"2024-05-03T20:32:15.372604+0000","last_active":"2024-05-03T20:34:35.595294+0000","last_peered":"2024-05-03T20:34:35.595294+0000","last_clean":"2024-05-03T20:34:35.595294+0000","last_became_active":"2024-05-03T20:32:15.366768+0000","last_became_peered":"2024-05-03T20:32:15.366768+0000","last_unstale":"2024-05-03T20:34:35.595294+0000","last_undegraded":"2024-05-03T20:34:35.595294+0000","last_fullsized":"2024-05-03T20:34:35.595294+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:29:02.045902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:29:02.045902+0000","last_clean_scrub_stamp":"2024-05-03T20:29:02.045902+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:02:05.424270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.931159+0000","last_change":"2024-05-03T20:33:40.710085+0000","last_active":"2024-05-03T20:34:35.931159+0000","last_peered":"2024-05-03T20:34:35.931159+0000","last_clean":"2024-05-03T20:34:35.931159+0000","last_became_active":"2024-05-03T20:33:40.709765+0000","last_became_peered":"2024-05-03T20:33:40.709765+0000","last_unstale":"2024-05-03T20:34:35.931159+0000","last_undegraded":"2024-05-03T20:34:35.931159+0000","last_fullsized":"2024-05-03T20:34:35.931159+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:52:02.339529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.603790+0000","last_change":"2024-05-03T20:34:35.603790+0000","last_active":"2024-05-03T20:34:35.603790+0000","last_peered":"2024-05-03T20:34:35.603790+0000","last_clean":"2024-05-03T20:34:35.603790+0000","last_became_active":"2024-05-03T20:34:35.600716+0000","last_became_peered":"2024-05-03T20:34:35.600716+0000","last_unstale":"2024-05-03T20:34:35.603790+0000","last_undegraded":"2024-05-03T20:34:35.603790+0000","last_fullsized":"2024-05-03T20:34:35.603790+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:30:02.768736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598276+0000","last_change":"2024-05-03T20:33:38.239293+0000","last_active":"2024-05-03T20:34:35.598276+0000","last_peered":"2024-05-03T20:34:35.598276+0000","last_clean":"2024-05-03T20:34:35.598276+0000","last_became_active":"2024-05-03T20:33:38.238888+0000","last_became_peered":"2024-05-03T20:33:38.238888+0000","last_unstale":"2024-05-03T20:34:35.598276+0000","last_undegraded":"2024-05-03T20:34:35.598276+0000","last_fullsized":"2024-05-03T20:34:35.598276+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:41:06.397280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.931490+0000","last_change":"2024-05-03T20:33:40.710180+0000","last_active":"2024-05-03T20:34:35.931490+0000","last_peered":"2024-05-03T20:34:35.931490+0000","last_clean":"2024-05-03T20:34:35.931490+0000","last_became_active":"2024-05-03T20:33:40.709322+0000","last_became_peered":"2024-05-03T20:33:40.709322+0000","last_unstale":"2024-05-03T20:34:35.931490+0000","last_undegraded":"2024-05-03T20:34:35.931490+0000","last_fullsized":"2024-05-03T20:34:35.931490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:15:44.569110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'13","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.591316+0000","last_change":"2024-05-03T20:33:37.682422+0000","last_active":"2024-05-03T20:34:35.591316+0000","last_peered":"2024-05-03T20:34:35.591316+0000","last_clean":"2024-05-03T20:34:35.591316+0000","last_became_active":"2024-05-03T20:33:37.682079+0000","last_became_peered":"2024-05-03T20:33:37.682079+0000","last_unstale":"2024-05-03T20:34:35.591316+0000","last_undegraded":"2024-05-03T20:34:35.591316+0000","last_fullsized":"2024-05-03T20:34:35.591316+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:54:43.806768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.789741+0000","last_change":"2024-05-03T20:33:40.715859+0000","last_active":"2024-05-03T20:34:35.789741+0000","last_peered":"2024-05-03T20:34:35.789741+0000","last_clean":"2024-05-03T20:34:35.789741+0000","last_became_active":"2024-05-03T20:33:40.715511+0000","last_became_peered":"2024-05-03T20:33:40.715511+0000","last_unstale":"2024-05-03T20:34:35.789741+0000","last_undegraded":"2024-05-03T20:34:35.789741+0000","last_fullsized":"2024-05-03T20:34:35.789741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:27:59.317403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.592182+0000","last_change":"2024-05-03T20:33:37.679161+0000","last_active":"2024-05-03T20:34:35.592182+0000","last_peered":"2024-05-03T20:34:35.592182+0000","last_clean":"2024-05-03T20:34:35.592182+0000","last_became_active":"2024-05-03T20:33:37.678825+0000","last_became_peered":"2024-05-03T20:33:37.678825+0000","last_unstale":"2024-05-03T20:34:35.592182+0000","last_undegraded":"2024-05-03T20:34:35.592182+0000","last_fullsized":"2024-05-03T20:34:35.592182+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:05:01.203880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.594073+0000","last_change":"2024-05-03T20:33:40.713287+0000","last_active":"2024-05-03T20:34:35.594073+0000","last_peered":"2024-05-03T20:34:35.594073+0000","last_clean":"2024-05-03T20:34:35.594073+0000","last_became_active":"2024-05-03T20:33:40.712962+0000","last_became_peered":"2024-05-03T20:33:40.712962+0000","last_unstale":"2024-05-03T20:34:35.594073+0000","last_undegraded":"2024-05-03T20:34:35.594073+0000","last_fullsized":"2024-05-03T20:34:35.594073+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:22:59.151645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598036+0000","last_change":"2024-05-03T20:33:38.237996+0000","last_active":"2024-05-03T20:34:35.598036+0000","last_peered":"2024-05-03T20:34:35.598036+0000","last_clean":"2024-05-03T20:34:35.598036+0000","last_became_active":"2024-05-03T20:33:38.237552+0000","last_became_peered":"2024-05-03T20:33:38.237552+0000","last_unstale":"2024-05-03T20:34:35.598036+0000","last_undegraded":"2024-05-03T20:34:35.598036+0000","last_fullsized":"2024-05-03T20:34:35.598036+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:23:47.946259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.597646+0000","last_change":"2024-05-03T20:33:40.709634+0000","last_active":"2024-05-03T20:34:35.597646+0000","last_peered":"2024-05-03T20:34:35.597646+0000","last_clean":"2024-05-03T20:34:35.597646+0000","last_became_active":"2024-05-03T20:33:40.709195+0000","last_became_peered":"2024-05-03T20:33:40.709195+0000","last_unstale":"2024-05-03T20:34:35.597646+0000","last_undegraded":"2024-05-03T20:34:35.597646+0000","last_fullsized":"2024-05-03T20:34:35.597646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:24:30.796765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.789390+0000","last_change":"2024-05-03T20:33:37.677729+0000","last_active":"2024-05-03T20:34:35.789390+0000","last_peered":"2024-05-03T20:34:35.789390+0000","last_clean":"2024-05-03T20:34:35.789390+0000","last_became_active":"2024-05-03T20:33:37.677285+0000","last_became_peered":"2024-05-03T20:33:37.677285+0000","last_unstale":"2024-05-03T20:34:35.789390+0000","last_undegraded":"2024-05-03T20:34:35.789390+0000","last_fullsized":"2024-05-03T20:34:35.789390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:01:46.289588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.594955+0000","last_change":"2024-05-03T20:33:40.708573+0000","last_active":"2024-05-03T20:34:35.594955+0000","last_peered":"2024-05-03T20:34:35.594955+0000","last_clean":"2024-05-03T20:34:35.594955+0000","last_became_active":"2024-05-03T20:33:40.708238+0000","last_became_peered":"2024-05-03T20:33:40.708238+0000","last_unstale":"2024-05-03T20:34:35.594955+0000","last_undegraded":"2024-05-03T20:34:35.594955+0000","last_fullsized":"2024-05-03T20:34:35.594955+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:21:06.975075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.593876+0000","last_change":"2024-05-03T20:33:37.683415+0000","last_active":"2024-05-03T20:34:35.593876+0000","last_peered":"2024-05-03T20:34:35.593876+0000","last_clean":"2024-05-03T20:34:35.593876+0000","last_became_active":"2024-05-03T20:33:37.682935+0000","last_became_peered":"2024-05-03T20:33:37.682935+0000","last_unstale":"2024-05-03T20:34:35.593876+0000","last_undegraded":"2024-05-03T20:34:35.593876+0000","last_fullsized":"2024-05-03T20:34:35.593876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:26:17.977777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.789573+0000","last_change":"2024-05-03T20:33:40.708116+0000","last_active":"2024-05-03T20:34:35.789573+0000","last_peered":"2024-05-03T20:34:35.789573+0000","last_clean":"2024-05-03T20:34:35.789573+0000","last_became_active":"2024-05-03T20:33:40.707625+0000","last_became_peered":"2024-05-03T20:33:40.707625+0000","last_unstale":"2024-05-03T20:34:35.789573+0000","last_undegraded":"2024-05-03T20:34:35.789573+0000","last_fullsized":"2024-05-03T20:34:35.789573+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:52:44.984252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598194+0000","last_change":"2024-05-03T20:33:37.685132+0000","last_active":"2024-05-03T20:34:35.598194+0000","last_peered":"2024-05-03T20:34:35.598194+0000","last_clean":"2024-05-03T20:34:35.598194+0000","last_became_active":"2024-05-03T20:33:37.684169+0000","last_became_peered":"2024-05-03T20:33:37.684169+0000","last_unstale":"2024-05-03T20:34:35.598194+0000","last_undegraded":"2024-05-03T20:34:35.598194+0000","last_fullsized":"2024-05-03T20:34:35.598194+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:41:24.163383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.595285+0000","last_change":"2024-05-03T20:33:40.713622+0000","last_active":"2024-05-03T20:34:35.595285+0000","last_peered":"2024-05-03T20:34:35.595285+0000","last_clean":"2024-05-03T20:34:35.595285+0000","last_became_active":"2024-05-03T20:33:40.712467+0000","last_became_peered":"2024-05-03T20:33:40.712467+0000","last_unstale":"2024-05-03T20:34:35.595285+0000","last_undegraded":"2024-05-03T20:34:35.595285+0000","last_fullsized":"2024-05-03T20:34:35.595285+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:40:07.948287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'5","reported_seq":71,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:35:25.672061+0000","last_change":"2024-05-03T20:34:35.604222+0000","last_active":"2024-05-03T20:35:25.672061+0000","last_peered":"2024-05-03T20:35:25.672061+0000","last_clean":"2024-05-03T20:35:25.672061+0000","last_became_active":"2024-05-03T20:34:35.597789+0000","last_became_peered":"2024-05-03T20:34:35.597789+0000","last_unstale":"2024-05-03T20:35:25.672061+0000","last_undegraded":"2024-05-03T20:35:25.672061+0000","last_fullsized":"2024-05-03T20:35:25.672061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:43:39.527278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.596430+0000","last_change":"2024-05-03T20:33:40.710127+0000","last_active":"2024-05-03T20:34:35.596430+0000","last_peered":"2024-05-03T20:34:35.596430+0000","last_clean":"2024-05-03T20:34:35.596430+0000","last_became_active":"2024-05-03T20:33:40.709743+0000","last_became_peered":"2024-05-03T20:33:40.709743+0000","last_unstale":"2024-05-03T20:34:35.596430+0000","last_undegraded":"2024-05-03T20:34:35.596430+0000","last_fullsized":"2024-05-03T20:34:35.596430+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:01:48.638259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598938+0000","last_change":"2024-05-03T20:33:37.680009+0000","last_active":"2024-05-03T20:34:35.598938+0000","last_peered":"2024-05-03T20:34:35.598938+0000","last_clean":"2024-05-03T20:34:35.598938+0000","last_became_active":"2024-05-03T20:33:37.679577+0000","last_became_peered":"2024-05-03T20:33:37.679577+0000","last_unstale":"2024-05-03T20:34:35.598938+0000","last_undegraded":"2024-05-03T20:34:35.598938+0000","last_fullsized":"2024-05-03T20:34:35.598938+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:13:01.801473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.595992+0000","last_change":"2024-05-03T20:33:40.714713+0000","last_active":"2024-05-03T20:34:35.595992+0000","last_peered":"2024-05-03T20:34:35.595992+0000","last_clean":"2024-05-03T20:34:35.595992+0000","last_became_active":"2024-05-03T20:33:40.711743+0000","last_became_peered":"2024-05-03T20:33:40.711743+0000","last_unstale":"2024-05-03T20:34:35.595992+0000","last_undegraded":"2024-05-03T20:34:35.595992+0000","last_fullsized":"2024-05-03T20:34:35.595992+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:57:42.748953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.931542+0000","last_change":"2024-05-03T20:33:37.682038+0000","last_active":"2024-05-03T20:34:35.931542+0000","last_peered":"2024-05-03T20:34:35.931542+0000","last_clean":"2024-05-03T20:34:35.931542+0000","last_became_active":"2024-05-03T20:33:37.681559+0000","last_became_peered":"2024-05-03T20:33:37.681559+0000","last_unstale":"2024-05-03T20:34:35.931542+0000","last_undegraded":"2024-05-03T20:34:35.931542+0000","last_fullsized":"2024-05-03T20:34:35.931542+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:12:49.532764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.596370+0000","last_change":"2024-05-03T20:33:40.708025+0000","last_active":"2024-05-03T20:34:35.596370+0000","last_peered":"2024-05-03T20:34:35.596370+0000","last_clean":"2024-05-03T20:34:35.596370+0000","last_became_active":"2024-05-03T20:33:40.707601+0000","last_became_peered":"2024-05-03T20:33:40.707601+0000","last_unstale":"2024-05-03T20:34:35.596370+0000","last_undegraded":"2024-05-03T20:34:35.596370+0000","last_fullsized":"2024-05-03T20:34:35.596370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:34:32.060247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.595077+0000","last_change":"2024-05-03T20:33:40.711728+0000","last_active":"2024-05-03T20:34:35.595077+0000","last_peered":"2024-05-03T20:34:35.595077+0000","last_clean":"2024-05-03T20:34:35.595077+0000","last_became_active":"2024-05-03T20:33:40.711367+0000","last_became_peered":"2024-05-03T20:33:40.711367+0000","last_unstale":"2024-05-03T20:34:35.595077+0000","last_undegraded":"2024-05-03T20:34:35.595077+0000","last_fullsized":"2024-05-03T20:34:35.595077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:07:17.956339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.933014+0000","last_change":"2024-05-03T20:34:35.933014+0000","last_active":"2024-05-03T20:34:35.933014+0000","last_peered":"2024-05-03T20:34:35.933014+0000","last_clean":"2024-05-03T20:34:35.933014+0000","last_became_active":"2024-05-03T20:34:35.932407+0000","last_became_peered":"2024-05-03T20:34:35.932407+0000","last_unstale":"2024-05-03T20:34:35.933014+0000","last_undegraded":"2024-05-03T20:34:35.933014+0000","last_fullsized":"2024-05-03T20:34:35.933014+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:09:46.789627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.593591+0000","last_change":"2024-05-03T20:33:40.705468+0000","last_active":"2024-05-03T20:34:35.593591+0000","last_peered":"2024-05-03T20:34:35.593591+0000","last_clean":"2024-05-03T20:34:35.593591+0000","last_became_active":"2024-05-03T20:33:40.705114+0000","last_became_peered":"2024-05-03T20:33:40.705114+0000","last_unstale":"2024-05-03T20:34:35.593591+0000","last_undegraded":"2024-05-03T20:34:35.593591+0000","last_fullsized":"2024-05-03T20:34:35.593591+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:20:05.549638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.594111+0000","last_change":"2024-05-03T20:33:40.713282+0000","last_active":"2024-05-03T20:34:35.594111+0000","last_peered":"2024-05-03T20:34:35.594111+0000","last_clean":"2024-05-03T20:34:35.594111+0000","last_became_active":"2024-05-03T20:33:40.712674+0000","last_became_peered":"2024-05-03T20:33:40.712674+0000","last_unstale":"2024-05-03T20:34:35.594111+0000","last_undegraded":"2024-05-03T20:34:35.594111+0000","last_fullsized":"2024-05-03T20:34:35.594111+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:46:53.034439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.931576+0000","last_change":"2024-05-03T20:33:40.707870+0000","last_active":"2024-05-03T20:34:35.931576+0000","last_peered":"2024-05-03T20:34:35.931576+0000","last_clean":"2024-05-03T20:34:35.931576+0000","last_became_active":"2024-05-03T20:33:40.706849+0000","last_became_peered":"2024-05-03T20:33:40.706849+0000","last_unstale":"2024-05-03T20:34:35.931576+0000","last_undegraded":"2024-05-03T20:34:35.931576+0000","last_fullsized":"2024-05-03T20:34:35.931576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:23:18.030508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.596075+0000","last_change":"2024-05-03T20:33:40.716365+0000","last_active":"2024-05-03T20:34:35.596075+0000","last_peered":"2024-05-03T20:34:35.596075+0000","last_clean":"2024-05-03T20:34:35.596075+0000","last_became_active":"2024-05-03T20:33:40.716010+0000","last_became_peered":"2024-05-03T20:33:40.716010+0000","last_unstale":"2024-05-03T20:34:35.596075+0000","last_undegraded":"2024-05-03T20:34:35.596075+0000","last_fullsized":"2024-05-03T20:34:35.596075+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:29:08.459822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.594842+0000","last_change":"2024-05-03T20:33:40.712105+0000","last_active":"2024-05-03T20:34:35.594842+0000","last_peered":"2024-05-03T20:34:35.594842+0000","last_clean":"2024-05-03T20:34:35.594842+0000","last_became_active":"2024-05-03T20:33:40.711753+0000","last_became_peered":"2024-05-03T20:33:40.711753+0000","last_unstale":"2024-05-03T20:34:35.594842+0000","last_undegraded":"2024-05-03T20:34:35.594842+0000","last_fullsized":"2024-05-03T20:34:35.594842+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:29:14.895475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.789838+0000","last_change":"2024-05-03T20:33:40.713143+0000","last_active":"2024-05-03T20:34:35.789838+0000","last_peered":"2024-05-03T20:34:35.789838+0000","last_clean":"2024-05-03T20:34:35.789838+0000","last_became_active":"2024-05-03T20:33:40.712776+0000","last_became_peered":"2024-05-03T20:33:40.712776+0000","last_unstale":"2024-05-03T20:34:35.789838+0000","last_undegraded":"2024-05-03T20:34:35.789838+0000","last_fullsized":"2024-05-03T20:34:35.789838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:43:23.345103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598063+0000","last_change":"2024-05-03T20:33:40.706181+0000","last_active":"2024-05-03T20:34:35.598063+0000","last_peered":"2024-05-03T20:34:35.598063+0000","last_clean":"2024-05-03T20:34:35.598063+0000","last_became_active":"2024-05-03T20:33:40.705808+0000","last_became_peered":"2024-05-03T20:33:40.705808+0000","last_unstale":"2024-05-03T20:34:35.598063+0000","last_undegraded":"2024-05-03T20:34:35.598063+0000","last_fullsized":"2024-05-03T20:34:35.598063+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:26:00.035916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598274+0000","last_change":"2024-05-03T20:34:35.597903+0000","last_active":"2024-05-03T20:34:35.598274+0000","last_peered":"2024-05-03T20:34:35.598274+0000","last_clean":"2024-05-03T20:34:35.598274+0000","last_became_active":"2024-05-03T20:34:35.597570+0000","last_became_peered":"2024-05-03T20:34:35.597570+0000","last_unstale":"2024-05-03T20:34:35.598274+0000","last_undegraded":"2024-05-03T20:34:35.598274+0000","last_fullsized":"2024-05-03T20:34:35.598274+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:56:32.480374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598307+0000","last_change":"2024-05-03T20:33:40.715949+0000","last_active":"2024-05-03T20:34:35.598307+0000","last_peered":"2024-05-03T20:34:35.598307+0000","last_clean":"2024-05-03T20:34:35.598307+0000","last_became_active":"2024-05-03T20:33:40.715366+0000","last_became_peered":"2024-05-03T20:33:40.715366+0000","last_unstale":"2024-05-03T20:34:35.598307+0000","last_undegraded":"2024-05-03T20:34:35.598307+0000","last_fullsized":"2024-05-03T20:34:35.598307+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:32:54.485990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.931437+0000","last_change":"2024-05-03T20:33:40.710447+0000","last_active":"2024-05-03T20:34:35.931437+0000","last_peered":"2024-05-03T20:34:35.931437+0000","last_clean":"2024-05-03T20:34:35.931437+0000","last_became_active":"2024-05-03T20:33:40.710107+0000","last_became_peered":"2024-05-03T20:33:40.710107+0000","last_unstale":"2024-05-03T20:34:35.931437+0000","last_undegraded":"2024-05-03T20:34:35.931437+0000","last_fullsized":"2024-05-03T20:34:35.931437+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:47:43.678947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.595647+0000","last_change":"2024-05-03T20:33:40.714081+0000","last_active":"2024-05-03T20:34:35.595647+0000","last_peered":"2024-05-03T20:34:35.595647+0000","last_clean":"2024-05-03T20:34:35.595647+0000","last_became_active":"2024-05-03T20:33:40.713678+0000","last_became_peered":"2024-05-03T20:33:40.713678+0000","last_unstale":"2024-05-03T20:34:35.595647+0000","last_undegraded":"2024-05-03T20:34:35.595647+0000","last_fullsized":"2024-05-03T20:34:35.595647+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:24:30.128065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60760,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":906,"num_read_kb":541,"num_write":241,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1170,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":182280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612645,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":438259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808874,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1593344,"data_stored":1082844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972401,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":438189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168629,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":921600,"data_stored":438567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397561,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28508,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716644,"statfs":{"total":95995035648,"available":95965843456,"internally_reserved":0,"allocated":1519616,"data_stored":1031867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561087,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":438297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":962560,"data_stored":492375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":929792,"data_stored":439245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":974848,"data_stored":491425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117069,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":925696,"data_stored":437731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":439328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1531904,"data_stored":1030763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":55308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":54358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":55292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-03T20:35:33.244 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-03T20:35:33.485 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:35:33.753 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:33 smithi181 ceph-mon[29801]: pgmap v342: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:35:33.753 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:33 smithi181 ceph-mon[29801]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T20:35:33.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:33 smithi196 ceph-mon[32858]: pgmap v342: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:35:33.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:33 smithi196 ceph-mon[32858]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T20:35:33.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:33 smithi204 ceph-mon[33595]: pgmap v342: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:35:33.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:33 smithi204 ceph-mon[33595]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T20:35:35.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.239+0000 7f8e3b07b700 1 -- 172.21.15.181:0/2982865783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e34102630 msgr2=0x7f8e34102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:35.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.239+0000 7f8e3b07b700 1 --2- 172.21.15.181:0/2982865783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e34102630 0x7f8e34102a10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8e280099c0 tx=0x7f8e28009cd0 comp rx=0 tx=0).stop 2024-05-03T20:35:35.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.240+0000 7f8e3b07b700 1 -- 172.21.15.181:0/2982865783 shutdown_connections 2024-05-03T20:35:35.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.240+0000 7f8e3b07b700 1 --2- 172.21.15.181:0/2982865783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e341076c0 0x7f8e3410f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:35.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.240+0000 7f8e3b07b700 1 --2- 172.21.15.181:0/2982865783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e34102630 0x7f8e34102a10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:35.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.240+0000 7f8e3b07b700 1 --2- 172.21.15.181:0/2982865783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e34102fe0 0x7f8e34106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:35.240 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.240+0000 7f8e3b07b700 1 -- 172.21.15.181:0/2982865783 >> 172.21.15.181:0/2982865783 conn(0x7f8e34079350 msgr2=0x7f8e34079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:35.240 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.240+0000 7f8e3b07b700 1 -- 172.21.15.181:0/2982865783 shutdown_connections 2024-05-03T20:35:35.240 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.240+0000 7f8e3b07b700 1 -- 172.21.15.181:0/2982865783 wait complete. 2024-05-03T20:35:35.241 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.240+0000 7f8e3b07b700 1 Processor -- start 2024-05-03T20:35:35.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.240+0000 7f8e3b07b700 1 -- start start 2024-05-03T20:35:35.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.241+0000 7f8e3b07b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e34102630 0x7f8e3419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:35.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.241+0000 7f8e3b07b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e34102fe0 0x7f8e3419afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:35.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.241+0000 7f8e3b07b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e341076c0 0x7f8e341a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:35.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.241+0000 7f8e3b07b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8e34112550 con 0x7f8e34102fe0 2024-05-03T20:35:35.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.241+0000 7f8e3b07b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e341123d0 con 0x7f8e34102630 2024-05-03T20:35:35.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.241+0000 7f8e3b07b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8e341126d0 con 0x7f8e341076c0 2024-05-03T20:35:35.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.241+0000 7f8e33fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e34102fe0 0x7f8e3419afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:35.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.241+0000 7f8e33fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e34102fe0 0x7f8e3419afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57492/0 (socket says 172.21.15.181:57492) 2024-05-03T20:35:35.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.241+0000 7f8e33fff700 1 -- 172.21.15.181:0/1587028668 learned_addr learned my addr 172.21.15.181:0/1587028668 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:35.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.241+0000 7f8e38e17700 1 --2- 172.21.15.181:0/1587028668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e34102630 0x7f8e3419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:35.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.241+0000 7f8e39618700 1 --2- 172.21.15.181:0/1587028668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e341076c0 0x7f8e341a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:35.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.241+0000 7f8e33fff700 1 -- 172.21.15.181:0/1587028668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e341076c0 msgr2=0x7f8e341a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:35.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.241+0000 7f8e33fff700 1 --2- 172.21.15.181:0/1587028668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e341076c0 0x7f8e341a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:35.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.241+0000 7f8e33fff700 1 -- 172.21.15.181:0/1587028668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e34102630 msgr2=0x7f8e3419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:35.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.241+0000 7f8e33fff700 1 --2- 172.21.15.181:0/1587028668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e34102630 0x7f8e3419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:35.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.241+0000 7f8e33fff700 1 -- 172.21.15.181:0/1587028668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e341a6030 con 0x7f8e34102fe0 2024-05-03T20:35:35.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.241+0000 7f8e33fff700 1 --2- 172.21.15.181:0/1587028668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e34102fe0 0x7f8e3419afb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8e2000dac0 tx=0x7f8e2000ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:35.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.242+0000 7f8e31ffb700 1 -- 172.21.15.181:0/1587028668 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e20017070 con 0x7f8e34102fe0 2024-05-03T20:35:35.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.242+0000 7f8e3b07b700 1 -- 172.21.15.181:0/1587028668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e341a6260 con 0x7f8e34102fe0 2024-05-03T20:35:35.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.242+0000 7f8e31ffb700 1 -- 172.21.15.181:0/1587028668 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e20009d80 con 0x7f8e34102fe0 2024-05-03T20:35:35.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.242+0000 7f8e31ffb700 1 -- 172.21.15.181:0/1587028668 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2000f3c0 con 0x7f8e34102fe0 2024-05-03T20:35:35.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.242+0000 7f8e3b07b700 1 -- 172.21.15.181:0/1587028668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e341a67f0 con 0x7f8e34102fe0 2024-05-03T20:35:35.244 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.243+0000 7f8e3b07b700 1 -- 172.21.15.181:0/1587028668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e3406a190 con 0x7f8e34102fe0 2024-05-03T20:35:35.244 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.244+0000 7f8e31ffb700 1 -- 172.21.15.181:0/1587028668 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f8e200098d0 con 0x7f8e34102fe0 2024-05-03T20:35:35.244 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.245+0000 7f8e31ffb700 1 --2- 172.21.15.181:0/1587028668 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8e240b2990 0x7f8e240b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:35.244 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.245+0000 7f8e38e17700 1 --2- 172.21.15.181:0/1587028668 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8e240b2990 0x7f8e240b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:35.244 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.245+0000 7f8e31ffb700 1 -- 172.21.15.181:0/1587028668 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f8e200e1740 con 0x7f8e34102fe0 2024-05-03T20:35:35.245 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.245+0000 7f8e38e17700 1 --2- 172.21.15.181:0/1587028668 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8e240b2990 0x7f8e240b4e50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8e28009990 tx=0x7f8e28005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:35.248 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.249+0000 7f8e31ffb700 1 -- 172.21.15.181:0/1587028668 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8e200a9300 con 0x7f8e34102fe0 2024-05-03T20:35:35.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.445+0000 7f8e3b07b700 1 -- 172.21.15.181:0/1587028668 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8e340638e0 con 0x7f8e240b2990 2024-05-03T20:35:35.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.452+0000 7f8e31ffb700 1 -- 172.21.15.181:0/1587028668 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258768 (secure 0 0 0) 0x7f8e340638e0 con 0x7f8e240b2990 2024-05-03T20:35:35.455 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:35:35.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.455+0000 7f8e3b07b700 1 -- 172.21.15.181:0/1587028668 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8e240b2990 msgr2=0x7f8e240b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:35.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.455+0000 7f8e3b07b700 1 --2- 172.21.15.181:0/1587028668 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8e240b2990 0x7f8e240b4e50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8e28009990 tx=0x7f8e28005c40 comp rx=0 tx=0).stop 2024-05-03T20:35:35.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.455+0000 7f8e3b07b700 1 -- 172.21.15.181:0/1587028668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e34102fe0 msgr2=0x7f8e3419afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:35.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.455+0000 7f8e3b07b700 1 --2- 172.21.15.181:0/1587028668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e34102fe0 0x7f8e3419afb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8e2000dac0 tx=0x7f8e2000ddd0 comp rx=0 tx=0).stop 2024-05-03T20:35:35.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.455+0000 7f8e3b07b700 1 -- 172.21.15.181:0/1587028668 shutdown_connections 2024-05-03T20:35:35.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.455+0000 7f8e3b07b700 1 --2- 172.21.15.181:0/1587028668 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8e240b2990 0x7f8e240b4e50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:35.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.455+0000 7f8e3b07b700 1 --2- 172.21.15.181:0/1587028668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e341076c0 0x7f8e341a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:35.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.455+0000 7f8e3b07b700 1 --2- 172.21.15.181:0/1587028668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e34102630 0x7f8e3419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:35.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.455+0000 7f8e3b07b700 1 --2- 172.21.15.181:0/1587028668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e34102fe0 0x7f8e3419afb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:35.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.455+0000 7f8e3b07b700 1 -- 172.21.15.181:0/1587028668 >> 172.21.15.181:0/1587028668 conn(0x7f8e34079350 msgr2=0x7f8e34100790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:35.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.456+0000 7f8e3b07b700 1 -- 172.21.15.181:0/1587028668 shutdown_connections 2024-05-03T20:35:35.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:35.456+0000 7f8e3b07b700 1 -- 172.21.15.181:0/1587028668 wait complete. 2024-05-03T20:35:35.459 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2024-05-03T20:35:35.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:35 smithi181 ceph-mon[29801]: pgmap v343: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:35:35.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:35 smithi196 ceph-mon[32858]: pgmap v343: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:35:35.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:35 smithi204 ceph-mon[33595]: pgmap v343: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:35:36.181 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-05-03T20:35:33.878485+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651362,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1017,"num_read_kb":620,"num_write":366,"num_write_kb":2280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2953010,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370256,"kb_used_data":12708,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571568,"statfs":{"total":1151940427776,"available":1151561285632,"internally_reserved":0,"allocated":13012992,"data_stored":7198890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003069"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.596646+0000","last_change":"2024-05-03T20:33:40.713047+0000","last_active":"2024-05-03T20:34:35.596646+0000","last_peered":"2024-05-03T20:34:35.596646+0000","last_clean":"2024-05-03T20:34:35.596646+0000","last_became_active":"2024-05-03T20:33:40.712718+0000","last_became_peered":"2024-05-03T20:33:40.712718+0000","last_unstale":"2024-05-03T20:34:35.596646+0000","last_undegraded":"2024-05-03T20:34:35.596646+0000","last_fullsized":"2024-05-03T20:34:35.596646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:55:24.816247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.594114+0000","last_change":"2024-05-03T20:33:40.716139+0000","last_active":"2024-05-03T20:34:35.594114+0000","last_peered":"2024-05-03T20:34:35.594114+0000","last_clean":"2024-05-03T20:34:35.594114+0000","last_became_active":"2024-05-03T20:33:40.715808+0000","last_became_peered":"2024-05-03T20:33:40.715808+0000","last_unstale":"2024-05-03T20:34:35.594114+0000","last_undegraded":"2024-05-03T20:34:35.594114+0000","last_fullsized":"2024-05-03T20:34:35.594114+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:36:30.410799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.593940+0000","last_change":"2024-05-03T20:33:40.712650+0000","last_active":"2024-05-03T20:34:35.593940+0000","last_peered":"2024-05-03T20:34:35.593940+0000","last_clean":"2024-05-03T20:34:35.593940+0000","last_became_active":"2024-05-03T20:33:40.712285+0000","last_became_peered":"2024-05-03T20:33:40.712285+0000","last_unstale":"2024-05-03T20:34:35.593940+0000","last_undegraded":"2024-05-03T20:34:35.593940+0000","last_fullsized":"2024-05-03T20:34:35.593940+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:10:25.162097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.592424+0000","last_change":"2024-05-03T20:33:40.716602+0000","last_active":"2024-05-03T20:34:35.592424+0000","last_peered":"2024-05-03T20:34:35.592424+0000","last_clean":"2024-05-03T20:34:35.592424+0000","last_became_active":"2024-05-03T20:33:40.716282+0000","last_became_peered":"2024-05-03T20:33:40.716282+0000","last_unstale":"2024-05-03T20:34:35.592424+0000","last_undegraded":"2024-05-03T20:34:35.592424+0000","last_fullsized":"2024-05-03T20:34:35.592424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:15:27.687359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598334+0000","last_change":"2024-05-03T20:33:40.701780+0000","last_active":"2024-05-03T20:34:35.598334+0000","last_peered":"2024-05-03T20:34:35.598334+0000","last_clean":"2024-05-03T20:34:35.598334+0000","last_became_active":"2024-05-03T20:33:40.701447+0000","last_became_peered":"2024-05-03T20:33:40.701447+0000","last_unstale":"2024-05-03T20:34:35.598334+0000","last_undegraded":"2024-05-03T20:34:35.598334+0000","last_fullsized":"2024-05-03T20:34:35.598334+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:23:53.219489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.594068+0000","last_change":"2024-05-03T20:33:40.721249+0000","last_active":"2024-05-03T20:34:35.594068+0000","last_peered":"2024-05-03T20:34:35.594068+0000","last_clean":"2024-05-03T20:34:35.594068+0000","last_became_active":"2024-05-03T20:33:40.720838+0000","last_became_peered":"2024-05-03T20:33:40.720838+0000","last_unstale":"2024-05-03T20:34:35.594068+0000","last_undegraded":"2024-05-03T20:34:35.594068+0000","last_fullsized":"2024-05-03T20:34:35.594068+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:37:12.868435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.595301+0000","last_change":"2024-05-03T20:33:40.708834+0000","last_active":"2024-05-03T20:34:35.595301+0000","last_peered":"2024-05-03T20:34:35.595301+0000","last_clean":"2024-05-03T20:34:35.595301+0000","last_became_active":"2024-05-03T20:33:40.708471+0000","last_became_peered":"2024-05-03T20:33:40.708471+0000","last_unstale":"2024-05-03T20:34:35.595301+0000","last_undegraded":"2024-05-03T20:34:35.595301+0000","last_fullsized":"2024-05-03T20:34:35.595301+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:54:21.297452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.789211+0000","last_change":"2024-05-03T20:33:40.707078+0000","last_active":"2024-05-03T20:34:35.789211+0000","last_peered":"2024-05-03T20:34:35.789211+0000","last_clean":"2024-05-03T20:34:35.789211+0000","last_became_active":"2024-05-03T20:33:40.706724+0000","last_became_peered":"2024-05-03T20:33:40.706724+0000","last_unstale":"2024-05-03T20:34:35.789211+0000","last_undegraded":"2024-05-03T20:34:35.789211+0000","last_fullsized":"2024-05-03T20:34:35.789211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:54:27.397305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.599112+0000","last_change":"2024-05-03T20:33:40.712580+0000","last_active":"2024-05-03T20:34:35.599112+0000","last_peered":"2024-05-03T20:34:35.599112+0000","last_clean":"2024-05-03T20:34:35.599112+0000","last_became_active":"2024-05-03T20:33:40.712252+0000","last_became_peered":"2024-05-03T20:33:40.712252+0000","last_unstale":"2024-05-03T20:34:35.599112+0000","last_undegraded":"2024-05-03T20:34:35.599112+0000","last_fullsized":"2024-05-03T20:34:35.599112+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:59:11.095956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.597225+0000","last_change":"2024-05-03T20:34:35.597225+0000","last_active":"2024-05-03T20:34:35.597225+0000","last_peered":"2024-05-03T20:34:35.597225+0000","last_clean":"2024-05-03T20:34:35.597225+0000","last_became_active":"2024-05-03T20:34:35.596786+0000","last_became_peered":"2024-05-03T20:34:35.596786+0000","last_unstale":"2024-05-03T20:34:35.597225+0000","last_undegraded":"2024-05-03T20:34:35.597225+0000","last_fullsized":"2024-05-03T20:34:35.597225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:04:14.637428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.595967+0000","last_change":"2024-05-03T20:33:40.711383+0000","last_active":"2024-05-03T20:34:35.595967+0000","last_peered":"2024-05-03T20:34:35.595967+0000","last_clean":"2024-05-03T20:34:35.595967+0000","last_became_active":"2024-05-03T20:33:40.710870+0000","last_became_peered":"2024-05-03T20:33:40.710870+0000","last_unstale":"2024-05-03T20:34:35.595967+0000","last_undegraded":"2024-05-03T20:34:35.595967+0000","last_fullsized":"2024-05-03T20:34:35.595967+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:49:18.905954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.592015+0000","last_change":"2024-05-03T20:33:40.713749+0000","last_active":"2024-05-03T20:34:35.592015+0000","last_peered":"2024-05-03T20:34:35.592015+0000","last_clean":"2024-05-03T20:34:35.592015+0000","last_became_active":"2024-05-03T20:33:40.713359+0000","last_became_peered":"2024-05-03T20:33:40.713359+0000","last_unstale":"2024-05-03T20:34:35.592015+0000","last_undegraded":"2024-05-03T20:34:35.592015+0000","last_fullsized":"2024-05-03T20:34:35.592015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:40:46.840267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.931512+0000","last_change":"2024-05-03T20:33:40.708434+0000","last_active":"2024-05-03T20:34:35.931512+0000","last_peered":"2024-05-03T20:34:35.931512+0000","last_clean":"2024-05-03T20:34:35.931512+0000","last_became_active":"2024-05-03T20:33:40.707825+0000","last_became_peered":"2024-05-03T20:33:40.707825+0000","last_unstale":"2024-05-03T20:34:35.931512+0000","last_undegraded":"2024-05-03T20:34:35.931512+0000","last_fullsized":"2024-05-03T20:34:35.931512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:42:49.837828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.593883+0000","last_change":"2024-05-03T20:33:40.714974+0000","last_active":"2024-05-03T20:34:35.593883+0000","last_peered":"2024-05-03T20:34:35.593883+0000","last_clean":"2024-05-03T20:34:35.593883+0000","last_became_active":"2024-05-03T20:33:40.714612+0000","last_became_peered":"2024-05-03T20:33:40.714612+0000","last_unstale":"2024-05-03T20:34:35.593883+0000","last_undegraded":"2024-05-03T20:34:35.593883+0000","last_fullsized":"2024-05-03T20:34:35.593883+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:34:40.766696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.593792+0000","last_change":"2024-05-03T20:33:40.713443+0000","last_active":"2024-05-03T20:34:35.593792+0000","last_peered":"2024-05-03T20:34:35.593792+0000","last_clean":"2024-05-03T20:34:35.593792+0000","last_became_active":"2024-05-03T20:33:40.712803+0000","last_became_peered":"2024-05-03T20:33:40.712803+0000","last_unstale":"2024-05-03T20:34:35.593792+0000","last_undegraded":"2024-05-03T20:34:35.593792+0000","last_fullsized":"2024-05-03T20:34:35.593792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:26:46.942918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.597473+0000","last_change":"2024-05-03T20:33:40.715667+0000","last_active":"2024-05-03T20:34:35.597473+0000","last_peered":"2024-05-03T20:34:35.597473+0000","last_clean":"2024-05-03T20:34:35.597473+0000","last_became_active":"2024-05-03T20:33:40.715291+0000","last_became_peered":"2024-05-03T20:33:40.715291+0000","last_unstale":"2024-05-03T20:34:35.597473+0000","last_undegraded":"2024-05-03T20:34:35.597473+0000","last_fullsized":"2024-05-03T20:34:35.597473+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:21:56.061022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.596902+0000","last_change":"2024-05-03T20:33:40.714495+0000","last_active":"2024-05-03T20:34:35.596902+0000","last_peered":"2024-05-03T20:34:35.596902+0000","last_clean":"2024-05-03T20:34:35.596902+0000","last_became_active":"2024-05-03T20:33:40.714176+0000","last_became_peered":"2024-05-03T20:33:40.714176+0000","last_unstale":"2024-05-03T20:34:35.596902+0000","last_undegraded":"2024-05-03T20:34:35.596902+0000","last_fullsized":"2024-05-03T20:34:35.596902+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:26:34.299414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.591286+0000","last_change":"2024-05-03T20:33:40.705512+0000","last_active":"2024-05-03T20:34:35.591286+0000","last_peered":"2024-05-03T20:34:35.591286+0000","last_clean":"2024-05-03T20:34:35.591286+0000","last_became_active":"2024-05-03T20:33:40.705187+0000","last_became_peered":"2024-05-03T20:33:40.705187+0000","last_unstale":"2024-05-03T20:34:35.591286+0000","last_undegraded":"2024-05-03T20:34:35.591286+0000","last_fullsized":"2024-05-03T20:34:35.591286+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:28:56.412004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.594429+0000","last_change":"2024-05-03T20:33:40.706780+0000","last_active":"2024-05-03T20:34:35.594429+0000","last_peered":"2024-05-03T20:34:35.594429+0000","last_clean":"2024-05-03T20:34:35.594429+0000","last_became_active":"2024-05-03T20:33:40.706439+0000","last_became_peered":"2024-05-03T20:33:40.706439+0000","last_unstale":"2024-05-03T20:34:35.594429+0000","last_undegraded":"2024-05-03T20:34:35.594429+0000","last_fullsized":"2024-05-03T20:34:35.594429+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:43:03.182219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":133,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:35:29.611412+0000","last_change":"2024-05-03T20:33:37.680899+0000","last_active":"2024-05-03T20:35:29.611412+0000","last_peered":"2024-05-03T20:35:29.611412+0000","last_clean":"2024-05-03T20:35:29.611412+0000","last_became_active":"2024-05-03T20:33:37.680602+0000","last_became_peered":"2024-05-03T20:33:37.680602+0000","last_unstale":"2024-05-03T20:35:29.611412+0000","last_undegraded":"2024-05-03T20:35:29.611412+0000","last_fullsized":"2024-05-03T20:35:29.611412+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:39:50.865486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.789426+0000","last_change":"2024-05-03T20:33:40.710869+0000","last_active":"2024-05-03T20:34:35.789426+0000","last_peered":"2024-05-03T20:34:35.789426+0000","last_clean":"2024-05-03T20:34:35.789426+0000","last_became_active":"2024-05-03T20:33:40.710552+0000","last_became_peered":"2024-05-03T20:33:40.710552+0000","last_unstale":"2024-05-03T20:34:35.789426+0000","last_undegraded":"2024-05-03T20:34:35.789426+0000","last_fullsized":"2024-05-03T20:34:35.789426+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:18:59.947702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.789757+0000","last_change":"2024-05-03T20:33:38.237387+0000","last_active":"2024-05-03T20:34:35.789757+0000","last_peered":"2024-05-03T20:34:35.789757+0000","last_clean":"2024-05-03T20:34:35.789757+0000","last_became_active":"2024-05-03T20:33:38.237029+0000","last_became_peered":"2024-05-03T20:33:38.237029+0000","last_unstale":"2024-05-03T20:34:35.789757+0000","last_undegraded":"2024-05-03T20:34:35.789757+0000","last_fullsized":"2024-05-03T20:34:35.789757+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:58:58.360973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.594278+0000","last_change":"2024-05-03T20:33:40.712653+0000","last_active":"2024-05-03T20:34:35.594278+0000","last_peered":"2024-05-03T20:34:35.594278+0000","last_clean":"2024-05-03T20:34:35.594278+0000","last_became_active":"2024-05-03T20:33:40.712283+0000","last_became_peered":"2024-05-03T20:33:40.712283+0000","last_unstale":"2024-05-03T20:34:35.594278+0000","last_undegraded":"2024-05-03T20:34:35.594278+0000","last_fullsized":"2024-05-03T20:34:35.594278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:28:56.388292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.611411+0000","last_change":"2024-05-03T20:34:35.610936+0000","last_active":"2024-05-03T20:34:35.611411+0000","last_peered":"2024-05-03T20:34:35.611411+0000","last_clean":"2024-05-03T20:34:35.611411+0000","last_became_active":"2024-05-03T20:34:35.597835+0000","last_became_peered":"2024-05-03T20:34:35.597835+0000","last_unstale":"2024-05-03T20:34:35.611411+0000","last_undegraded":"2024-05-03T20:34:35.611411+0000","last_fullsized":"2024-05-03T20:34:35.611411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:35:54.327195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598349+0000","last_change":"2024-05-03T20:33:40.711274+0000","last_active":"2024-05-03T20:34:35.598349+0000","last_peered":"2024-05-03T20:34:35.598349+0000","last_clean":"2024-05-03T20:34:35.598349+0000","last_became_active":"2024-05-03T20:33:40.710917+0000","last_became_peered":"2024-05-03T20:33:40.710917+0000","last_unstale":"2024-05-03T20:34:35.598349+0000","last_undegraded":"2024-05-03T20:34:35.598349+0000","last_fullsized":"2024-05-03T20:34:35.598349+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:22:55.969325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598238+0000","last_change":"2024-05-03T20:33:37.681383+0000","last_active":"2024-05-03T20:34:35.598238+0000","last_peered":"2024-05-03T20:34:35.598238+0000","last_clean":"2024-05-03T20:34:35.598238+0000","last_became_active":"2024-05-03T20:33:37.680941+0000","last_became_peered":"2024-05-03T20:33:37.680941+0000","last_unstale":"2024-05-03T20:34:35.598238+0000","last_undegraded":"2024-05-03T20:34:35.598238+0000","last_fullsized":"2024-05-03T20:34:35.598238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:03:26.083579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.599057+0000","last_change":"2024-05-03T20:33:40.711762+0000","last_active":"2024-05-03T20:34:35.599057+0000","last_peered":"2024-05-03T20:34:35.599057+0000","last_clean":"2024-05-03T20:34:35.599057+0000","last_became_active":"2024-05-03T20:33:40.710630+0000","last_became_peered":"2024-05-03T20:33:40.710630+0000","last_unstale":"2024-05-03T20:34:35.599057+0000","last_undegraded":"2024-05-03T20:34:35.599057+0000","last_fullsized":"2024-05-03T20:34:35.599057+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:42:23.728916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":128,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:35:30.614162+0000","last_change":"2024-05-03T20:33:37.685895+0000","last_active":"2024-05-03T20:35:30.614162+0000","last_peered":"2024-05-03T20:35:30.614162+0000","last_clean":"2024-05-03T20:35:30.614162+0000","last_became_active":"2024-05-03T20:33:37.685399+0000","last_became_peered":"2024-05-03T20:33:37.685399+0000","last_unstale":"2024-05-03T20:35:30.614162+0000","last_undegraded":"2024-05-03T20:35:30.614162+0000","last_fullsized":"2024-05-03T20:35:30.614162+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:09:29.932599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.597868+0000","last_change":"2024-05-03T20:33:40.704967+0000","last_active":"2024-05-03T20:34:35.597868+0000","last_peered":"2024-05-03T20:34:35.597868+0000","last_clean":"2024-05-03T20:34:35.597868+0000","last_became_active":"2024-05-03T20:33:40.704583+0000","last_became_peered":"2024-05-03T20:33:40.704583+0000","last_unstale":"2024-05-03T20:34:35.597868+0000","last_undegraded":"2024-05-03T20:34:35.597868+0000","last_fullsized":"2024-05-03T20:34:35.597868+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:46:31.720865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.593998+0000","last_change":"2024-05-03T20:33:37.680798+0000","last_active":"2024-05-03T20:34:35.593998+0000","last_peered":"2024-05-03T20:34:35.593998+0000","last_clean":"2024-05-03T20:34:35.593998+0000","last_became_active":"2024-05-03T20:33:37.680344+0000","last_became_peered":"2024-05-03T20:33:37.680344+0000","last_unstale":"2024-05-03T20:34:35.593998+0000","last_undegraded":"2024-05-03T20:34:35.593998+0000","last_fullsized":"2024-05-03T20:34:35.593998+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:26:32.037104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.791470+0000","last_change":"2024-05-03T20:34:35.790972+0000","last_active":"2024-05-03T20:34:35.791470+0000","last_peered":"2024-05-03T20:34:35.791470+0000","last_clean":"2024-05-03T20:34:35.791470+0000","last_became_active":"2024-05-03T20:34:35.790574+0000","last_became_peered":"2024-05-03T20:34:35.790574+0000","last_unstale":"2024-05-03T20:34:35.791470+0000","last_undegraded":"2024-05-03T20:34:35.791470+0000","last_fullsized":"2024-05-03T20:34:35.791470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:19:16.992618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.931345+0000","last_change":"2024-05-03T20:33:37.684230+0000","last_active":"2024-05-03T20:34:35.931345+0000","last_peered":"2024-05-03T20:34:35.931345+0000","last_clean":"2024-05-03T20:34:35.931345+0000","last_became_active":"2024-05-03T20:33:37.683844+0000","last_became_peered":"2024-05-03T20:33:37.683844+0000","last_unstale":"2024-05-03T20:34:35.931345+0000","last_undegraded":"2024-05-03T20:34:35.931345+0000","last_fullsized":"2024-05-03T20:34:35.931345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:46:23.417163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.789235+0000","last_change":"2024-05-03T20:33:40.706554+0000","last_active":"2024-05-03T20:34:35.789235+0000","last_peered":"2024-05-03T20:34:35.789235+0000","last_clean":"2024-05-03T20:34:35.789235+0000","last_became_active":"2024-05-03T20:33:40.706142+0000","last_became_peered":"2024-05-03T20:33:40.706142+0000","last_unstale":"2024-05-03T20:34:35.789235+0000","last_undegraded":"2024-05-03T20:34:35.789235+0000","last_fullsized":"2024-05-03T20:34:35.789235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:01:28.324024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.597925+0000","last_change":"2024-05-03T20:33:37.683497+0000","last_active":"2024-05-03T20:34:35.597925+0000","last_peered":"2024-05-03T20:34:35.597925+0000","last_clean":"2024-05-03T20:34:35.597925+0000","last_became_active":"2024-05-03T20:33:37.683178+0000","last_became_peered":"2024-05-03T20:33:37.683178+0000","last_unstale":"2024-05-03T20:34:35.597925+0000","last_undegraded":"2024-05-03T20:34:35.597925+0000","last_fullsized":"2024-05-03T20:34:35.597925+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:07:29.771786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.590905+0000","last_change":"2024-05-03T20:33:40.705447+0000","last_active":"2024-05-03T20:34:35.590905+0000","last_peered":"2024-05-03T20:34:35.590905+0000","last_clean":"2024-05-03T20:34:35.590905+0000","last_became_active":"2024-05-03T20:33:40.705102+0000","last_became_peered":"2024-05-03T20:33:40.705102+0000","last_unstale":"2024-05-03T20:34:35.590905+0000","last_undegraded":"2024-05-03T20:34:35.590905+0000","last_fullsized":"2024-05-03T20:34:35.590905+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:55:55.354498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":219,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:35:32.616213+0000","last_change":"2024-05-03T20:33:37.681449+0000","last_active":"2024-05-03T20:35:32.616213+0000","last_peered":"2024-05-03T20:35:32.616213+0000","last_clean":"2024-05-03T20:35:32.616213+0000","last_became_active":"2024-05-03T20:33:37.680798+0000","last_became_peered":"2024-05-03T20:33:37.680798+0000","last_unstale":"2024-05-03T20:35:32.616213+0000","last_undegraded":"2024-05-03T20:35:32.616213+0000","last_fullsized":"2024-05-03T20:35:32.616213+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:42:10.029261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598976+0000","last_change":"2024-05-03T20:33:40.708872+0000","last_active":"2024-05-03T20:34:35.598976+0000","last_peered":"2024-05-03T20:34:35.598976+0000","last_clean":"2024-05-03T20:34:35.598976+0000","last_became_active":"2024-05-03T20:33:40.708535+0000","last_became_peered":"2024-05-03T20:33:40.708535+0000","last_unstale":"2024-05-03T20:34:35.598976+0000","last_undegraded":"2024-05-03T20:34:35.598976+0000","last_fullsized":"2024-05-03T20:34:35.598976+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:21:08.949065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.938964+0000","last_change":"2024-05-03T20:34:35.938519+0000","last_active":"2024-05-03T20:34:35.938964+0000","last_peered":"2024-05-03T20:34:35.938964+0000","last_clean":"2024-05-03T20:34:35.938964+0000","last_became_active":"2024-05-03T20:34:35.932521+0000","last_became_peered":"2024-05-03T20:34:35.932521+0000","last_unstale":"2024-05-03T20:34:35.938964+0000","last_undegraded":"2024-05-03T20:34:35.938964+0000","last_fullsized":"2024-05-03T20:34:35.938964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:44:28.361138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,6],"acting":[11,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.594170+0000","last_change":"2024-05-03T20:33:40.713991+0000","last_active":"2024-05-03T20:34:35.594170+0000","last_peered":"2024-05-03T20:34:35.594170+0000","last_clean":"2024-05-03T20:34:35.594170+0000","last_became_active":"2024-05-03T20:33:40.713661+0000","last_became_peered":"2024-05-03T20:33:40.713661+0000","last_unstale":"2024-05-03T20:34:35.594170+0000","last_undegraded":"2024-05-03T20:34:35.594170+0000","last_fullsized":"2024-05-03T20:34:35.594170+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:08:20.818863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.599169+0000","last_change":"2024-05-03T20:33:38.237724+0000","last_active":"2024-05-03T20:34:35.599169+0000","last_peered":"2024-05-03T20:34:35.599169+0000","last_clean":"2024-05-03T20:34:35.599169+0000","last_became_active":"2024-05-03T20:33:38.237340+0000","last_became_peered":"2024-05-03T20:33:38.237340+0000","last_unstale":"2024-05-03T20:34:35.599169+0000","last_undegraded":"2024-05-03T20:34:35.599169+0000","last_fullsized":"2024-05-03T20:34:35.599169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:13:35.567527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598543+0000","last_change":"2024-05-03T20:33:40.707454+0000","last_active":"2024-05-03T20:34:35.598543+0000","last_peered":"2024-05-03T20:34:35.598543+0000","last_clean":"2024-05-03T20:34:35.598543+0000","last_became_active":"2024-05-03T20:33:40.707080+0000","last_became_peered":"2024-05-03T20:33:40.707080+0000","last_unstale":"2024-05-03T20:34:35.598543+0000","last_undegraded":"2024-05-03T20:34:35.598543+0000","last_fullsized":"2024-05-03T20:34:35.598543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:12:18.432048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'30","reported_seq":253,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:35:32.615860+0000","last_change":"2024-05-03T20:33:37.681144+0000","last_active":"2024-05-03T20:35:32.615860+0000","last_peered":"2024-05-03T20:35:32.615860+0000","last_clean":"2024-05-03T20:35:32.615860+0000","last_became_active":"2024-05-03T20:33:37.680531+0000","last_became_peered":"2024-05-03T20:33:37.680531+0000","last_unstale":"2024-05-03T20:35:32.615860+0000","last_undegraded":"2024-05-03T20:35:32.615860+0000","last_fullsized":"2024-05-03T20:35:32.615860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:45:53.536622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53165,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":198,"num_read_kb":150,"num_write":32,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598177+0000","last_change":"2024-05-03T20:33:40.711336+0000","last_active":"2024-05-03T20:34:35.598177+0000","last_peered":"2024-05-03T20:34:35.598177+0000","last_clean":"2024-05-03T20:34:35.598177+0000","last_became_active":"2024-05-03T20:33:40.710991+0000","last_became_peered":"2024-05-03T20:33:40.710991+0000","last_unstale":"2024-05-03T20:34:35.598177+0000","last_undegraded":"2024-05-03T20:34:35.598177+0000","last_fullsized":"2024-05-03T20:34:35.598177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:59:44.729974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.593972+0000","last_change":"2024-05-03T20:33:37.682280+0000","last_active":"2024-05-03T20:34:35.593972+0000","last_peered":"2024-05-03T20:34:35.593972+0000","last_clean":"2024-05-03T20:34:35.593972+0000","last_became_active":"2024-05-03T20:33:37.681612+0000","last_became_peered":"2024-05-03T20:33:37.681612+0000","last_unstale":"2024-05-03T20:34:35.593972+0000","last_undegraded":"2024-05-03T20:34:35.593972+0000","last_fullsized":"2024-05-03T20:34:35.593972+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:50:05.310560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.931335+0000","last_change":"2024-05-03T20:33:40.710309+0000","last_active":"2024-05-03T20:34:35.931335+0000","last_peered":"2024-05-03T20:34:35.931335+0000","last_clean":"2024-05-03T20:34:35.931335+0000","last_became_active":"2024-05-03T20:33:40.710015+0000","last_became_peered":"2024-05-03T20:33:40.710015+0000","last_unstale":"2024-05-03T20:34:35.931335+0000","last_undegraded":"2024-05-03T20:34:35.931335+0000","last_fullsized":"2024-05-03T20:34:35.931335+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:05:04.042635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598619+0000","last_change":"2024-05-03T20:33:37.679525+0000","last_active":"2024-05-03T20:34:35.598619+0000","last_peered":"2024-05-03T20:34:35.598619+0000","last_clean":"2024-05-03T20:34:35.598619+0000","last_became_active":"2024-05-03T20:33:37.678905+0000","last_became_peered":"2024-05-03T20:33:37.678905+0000","last_unstale":"2024-05-03T20:34:35.598619+0000","last_undegraded":"2024-05-03T20:34:35.598619+0000","last_fullsized":"2024-05-03T20:34:35.598619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:17:06.225255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.595860+0000","last_change":"2024-05-03T20:33:40.717613+0000","last_active":"2024-05-03T20:34:35.595860+0000","last_peered":"2024-05-03T20:34:35.595860+0000","last_clean":"2024-05-03T20:34:35.595860+0000","last_became_active":"2024-05-03T20:33:40.717348+0000","last_became_peered":"2024-05-03T20:33:40.717348+0000","last_unstale":"2024-05-03T20:34:35.595860+0000","last_undegraded":"2024-05-03T20:34:35.595860+0000","last_fullsized":"2024-05-03T20:34:35.595860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:20:31.493731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'6","reported_seq":129,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:35:30.613624+0000","last_change":"2024-05-03T20:33:37.683083+0000","last_active":"2024-05-03T20:35:30.613624+0000","last_peered":"2024-05-03T20:35:30.613624+0000","last_clean":"2024-05-03T20:35:30.613624+0000","last_became_active":"2024-05-03T20:33:37.682700+0000","last_became_peered":"2024-05-03T20:33:37.682700+0000","last_unstale":"2024-05-03T20:35:30.613624+0000","last_undegraded":"2024-05-03T20:35:30.613624+0000","last_fullsized":"2024-05-03T20:35:30.613624+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:47:18.294314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.595088+0000","last_change":"2024-05-03T20:33:40.711711+0000","last_active":"2024-05-03T20:34:35.595088+0000","last_peered":"2024-05-03T20:34:35.595088+0000","last_clean":"2024-05-03T20:34:35.595088+0000","last_became_active":"2024-05-03T20:33:40.711116+0000","last_became_peered":"2024-05-03T20:33:40.711116+0000","last_unstale":"2024-05-03T20:34:35.595088+0000","last_undegraded":"2024-05-03T20:34:35.595088+0000","last_fullsized":"2024-05-03T20:34:35.595088+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:53:18.231575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.597738+0000","last_change":"2024-05-03T20:33:38.238005+0000","last_active":"2024-05-03T20:34:35.597738+0000","last_peered":"2024-05-03T20:34:35.597738+0000","last_clean":"2024-05-03T20:34:35.597738+0000","last_became_active":"2024-05-03T20:33:38.237636+0000","last_became_peered":"2024-05-03T20:33:38.237636+0000","last_unstale":"2024-05-03T20:34:35.597738+0000","last_undegraded":"2024-05-03T20:34:35.597738+0000","last_fullsized":"2024-05-03T20:34:35.597738+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:49:15.501715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.599844+0000","last_change":"2024-05-03T20:33:40.716317+0000","last_active":"2024-05-03T20:34:35.599844+0000","last_peered":"2024-05-03T20:34:35.599844+0000","last_clean":"2024-05-03T20:34:35.599844+0000","last_became_active":"2024-05-03T20:33:40.715780+0000","last_became_peered":"2024-05-03T20:33:40.715780+0000","last_unstale":"2024-05-03T20:34:35.599844+0000","last_undegraded":"2024-05-03T20:34:35.599844+0000","last_fullsized":"2024-05-03T20:34:35.599844+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:21:46.186992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598597+0000","last_change":"2024-05-03T20:33:37.679690+0000","last_active":"2024-05-03T20:34:35.598597+0000","last_peered":"2024-05-03T20:34:35.598597+0000","last_clean":"2024-05-03T20:34:35.598597+0000","last_became_active":"2024-05-03T20:33:37.679214+0000","last_became_peered":"2024-05-03T20:33:37.679214+0000","last_unstale":"2024-05-03T20:34:35.598597+0000","last_undegraded":"2024-05-03T20:34:35.598597+0000","last_fullsized":"2024-05-03T20:34:35.598597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:08:54.467850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.595516+0000","last_change":"2024-05-03T20:33:40.714800+0000","last_active":"2024-05-03T20:34:35.595516+0000","last_peered":"2024-05-03T20:34:35.595516+0000","last_clean":"2024-05-03T20:34:35.595516+0000","last_became_active":"2024-05-03T20:33:40.714417+0000","last_became_peered":"2024-05-03T20:33:40.714417+0000","last_unstale":"2024-05-03T20:34:35.595516+0000","last_undegraded":"2024-05-03T20:34:35.595516+0000","last_fullsized":"2024-05-03T20:34:35.595516+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:06:37.110378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.591103+0000","last_change":"2024-05-03T20:33:38.237270+0000","last_active":"2024-05-03T20:34:35.591103+0000","last_peered":"2024-05-03T20:34:35.591103+0000","last_clean":"2024-05-03T20:34:35.591103+0000","last_became_active":"2024-05-03T20:33:38.236917+0000","last_became_peered":"2024-05-03T20:33:38.236917+0000","last_unstale":"2024-05-03T20:34:35.591103+0000","last_undegraded":"2024-05-03T20:34:35.591103+0000","last_fullsized":"2024-05-03T20:34:35.591103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:38:58.079905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.594215+0000","last_change":"2024-05-03T20:33:40.707368+0000","last_active":"2024-05-03T20:34:35.594215+0000","last_peered":"2024-05-03T20:34:35.594215+0000","last_clean":"2024-05-03T20:34:35.594215+0000","last_became_active":"2024-05-03T20:33:40.707049+0000","last_became_peered":"2024-05-03T20:33:40.707049+0000","last_unstale":"2024-05-03T20:34:35.594215+0000","last_undegraded":"2024-05-03T20:34:35.594215+0000","last_fullsized":"2024-05-03T20:34:35.594215+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:02:57.790142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'6","reported_seq":126,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:35:28.610728+0000","last_change":"2024-05-03T20:33:37.679007+0000","last_active":"2024-05-03T20:35:28.610728+0000","last_peered":"2024-05-03T20:35:28.610728+0000","last_clean":"2024-05-03T20:35:28.610728+0000","last_became_active":"2024-05-03T20:33:37.678609+0000","last_became_peered":"2024-05-03T20:33:37.678609+0000","last_unstale":"2024-05-03T20:35:28.610728+0000","last_undegraded":"2024-05-03T20:35:28.610728+0000","last_fullsized":"2024-05-03T20:35:28.610728+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:30:17.622905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.594121+0000","last_change":"2024-05-03T20:33:40.713820+0000","last_active":"2024-05-03T20:34:35.594121+0000","last_peered":"2024-05-03T20:34:35.594121+0000","last_clean":"2024-05-03T20:34:35.594121+0000","last_became_active":"2024-05-03T20:33:40.713487+0000","last_became_peered":"2024-05-03T20:33:40.713487+0000","last_unstale":"2024-05-03T20:34:35.594121+0000","last_undegraded":"2024-05-03T20:34:35.594121+0000","last_fullsized":"2024-05-03T20:34:35.594121+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:23:03.724180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598253+0000","last_change":"2024-05-03T20:33:40.716663+0000","last_active":"2024-05-03T20:34:35.598253+0000","last_peered":"2024-05-03T20:34:35.598253+0000","last_clean":"2024-05-03T20:34:35.598253+0000","last_became_active":"2024-05-03T20:33:40.716201+0000","last_became_peered":"2024-05-03T20:33:40.716201+0000","last_unstale":"2024-05-03T20:34:35.598253+0000","last_undegraded":"2024-05-03T20:34:35.598253+0000","last_fullsized":"2024-05-03T20:34:35.598253+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:38:12.975351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":125,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:35:30.613068+0000","last_change":"2024-05-03T20:33:38.237285+0000","last_active":"2024-05-03T20:35:30.613068+0000","last_peered":"2024-05-03T20:35:30.613068+0000","last_clean":"2024-05-03T20:35:30.613068+0000","last_became_active":"2024-05-03T20:33:38.236825+0000","last_became_peered":"2024-05-03T20:33:38.236825+0000","last_unstale":"2024-05-03T20:35:30.613068+0000","last_undegraded":"2024-05-03T20:35:30.613068+0000","last_fullsized":"2024-05-03T20:35:30.613068+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:46:52.963953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.789739+0000","last_change":"2024-05-03T20:33:40.713271+0000","last_active":"2024-05-03T20:34:35.789739+0000","last_peered":"2024-05-03T20:34:35.789739+0000","last_clean":"2024-05-03T20:34:35.789739+0000","last_became_active":"2024-05-03T20:33:40.712972+0000","last_became_peered":"2024-05-03T20:33:40.712972+0000","last_unstale":"2024-05-03T20:34:35.789739+0000","last_undegraded":"2024-05-03T20:34:35.789739+0000","last_fullsized":"2024-05-03T20:34:35.789739+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:39:46.762096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.591878+0000","last_change":"2024-05-03T20:33:37.682196+0000","last_active":"2024-05-03T20:34:35.591878+0000","last_peered":"2024-05-03T20:34:35.591878+0000","last_clean":"2024-05-03T20:34:35.591878+0000","last_became_active":"2024-05-03T20:33:37.681671+0000","last_became_peered":"2024-05-03T20:33:37.681671+0000","last_unstale":"2024-05-03T20:34:35.591878+0000","last_undegraded":"2024-05-03T20:34:35.591878+0000","last_fullsized":"2024-05-03T20:34:35.591878+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:53:26.866288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":102,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.595294+0000","last_change":"2024-05-03T20:32:15.372604+0000","last_active":"2024-05-03T20:34:35.595294+0000","last_peered":"2024-05-03T20:34:35.595294+0000","last_clean":"2024-05-03T20:34:35.595294+0000","last_became_active":"2024-05-03T20:32:15.366768+0000","last_became_peered":"2024-05-03T20:32:15.366768+0000","last_unstale":"2024-05-03T20:34:35.595294+0000","last_undegraded":"2024-05-03T20:34:35.595294+0000","last_fullsized":"2024-05-03T20:34:35.595294+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:29:02.045902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:29:02.045902+0000","last_clean_scrub_stamp":"2024-05-03T20:29:02.045902+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:02:05.424270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.931159+0000","last_change":"2024-05-03T20:33:40.710085+0000","last_active":"2024-05-03T20:34:35.931159+0000","last_peered":"2024-05-03T20:34:35.931159+0000","last_clean":"2024-05-03T20:34:35.931159+0000","last_became_active":"2024-05-03T20:33:40.709765+0000","last_became_peered":"2024-05-03T20:33:40.709765+0000","last_unstale":"2024-05-03T20:34:35.931159+0000","last_undegraded":"2024-05-03T20:34:35.931159+0000","last_fullsized":"2024-05-03T20:34:35.931159+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:52:02.339529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.603790+0000","last_change":"2024-05-03T20:34:35.603790+0000","last_active":"2024-05-03T20:34:35.603790+0000","last_peered":"2024-05-03T20:34:35.603790+0000","last_clean":"2024-05-03T20:34:35.603790+0000","last_became_active":"2024-05-03T20:34:35.600716+0000","last_became_peered":"2024-05-03T20:34:35.600716+0000","last_unstale":"2024-05-03T20:34:35.603790+0000","last_undegraded":"2024-05-03T20:34:35.603790+0000","last_fullsized":"2024-05-03T20:34:35.603790+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:30:02.768736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598276+0000","last_change":"2024-05-03T20:33:38.239293+0000","last_active":"2024-05-03T20:34:35.598276+0000","last_peered":"2024-05-03T20:34:35.598276+0000","last_clean":"2024-05-03T20:34:35.598276+0000","last_became_active":"2024-05-03T20:33:38.238888+0000","last_became_peered":"2024-05-03T20:33:38.238888+0000","last_unstale":"2024-05-03T20:34:35.598276+0000","last_undegraded":"2024-05-03T20:34:35.598276+0000","last_fullsized":"2024-05-03T20:34:35.598276+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:41:06.397280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.931490+0000","last_change":"2024-05-03T20:33:40.710180+0000","last_active":"2024-05-03T20:34:35.931490+0000","last_peered":"2024-05-03T20:34:35.931490+0000","last_clean":"2024-05-03T20:34:35.931490+0000","last_became_active":"2024-05-03T20:33:40.709322+0000","last_became_peered":"2024-05-03T20:33:40.709322+0000","last_unstale":"2024-05-03T20:34:35.931490+0000","last_undegraded":"2024-05-03T20:34:35.931490+0000","last_fullsized":"2024-05-03T20:34:35.931490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:15:44.569110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'13","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.591316+0000","last_change":"2024-05-03T20:33:37.682422+0000","last_active":"2024-05-03T20:34:35.591316+0000","last_peered":"2024-05-03T20:34:35.591316+0000","last_clean":"2024-05-03T20:34:35.591316+0000","last_became_active":"2024-05-03T20:33:37.682079+0000","last_became_peered":"2024-05-03T20:33:37.682079+0000","last_unstale":"2024-05-03T20:34:35.591316+0000","last_undegraded":"2024-05-03T20:34:35.591316+0000","last_fullsized":"2024-05-03T20:34:35.591316+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:54:43.806768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.789741+0000","last_change":"2024-05-03T20:33:40.715859+0000","last_active":"2024-05-03T20:34:35.789741+0000","last_peered":"2024-05-03T20:34:35.789741+0000","last_clean":"2024-05-03T20:34:35.789741+0000","last_became_active":"2024-05-03T20:33:40.715511+0000","last_became_peered":"2024-05-03T20:33:40.715511+0000","last_unstale":"2024-05-03T20:34:35.789741+0000","last_undegraded":"2024-05-03T20:34:35.789741+0000","last_fullsized":"2024-05-03T20:34:35.789741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:27:59.317403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.592182+0000","last_change":"2024-05-03T20:33:37.679161+0000","last_active":"2024-05-03T20:34:35.592182+0000","last_peered":"2024-05-03T20:34:35.592182+0000","last_clean":"2024-05-03T20:34:35.592182+0000","last_became_active":"2024-05-03T20:33:37.678825+0000","last_became_peered":"2024-05-03T20:33:37.678825+0000","last_unstale":"2024-05-03T20:34:35.592182+0000","last_undegraded":"2024-05-03T20:34:35.592182+0000","last_fullsized":"2024-05-03T20:34:35.592182+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:05:01.203880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.594073+0000","last_change":"2024-05-03T20:33:40.713287+0000","last_active":"2024-05-03T20:34:35.594073+0000","last_peered":"2024-05-03T20:34:35.594073+0000","last_clean":"2024-05-03T20:34:35.594073+0000","last_became_active":"2024-05-03T20:33:40.712962+0000","last_became_peered":"2024-05-03T20:33:40.712962+0000","last_unstale":"2024-05-03T20:34:35.594073+0000","last_undegraded":"2024-05-03T20:34:35.594073+0000","last_fullsized":"2024-05-03T20:34:35.594073+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:22:59.151645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598036+0000","last_change":"2024-05-03T20:33:38.237996+0000","last_active":"2024-05-03T20:34:35.598036+0000","last_peered":"2024-05-03T20:34:35.598036+0000","last_clean":"2024-05-03T20:34:35.598036+0000","last_became_active":"2024-05-03T20:33:38.237552+0000","last_became_peered":"2024-05-03T20:33:38.237552+0000","last_unstale":"2024-05-03T20:34:35.598036+0000","last_undegraded":"2024-05-03T20:34:35.598036+0000","last_fullsized":"2024-05-03T20:34:35.598036+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:23:47.946259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.597646+0000","last_change":"2024-05-03T20:33:40.709634+0000","last_active":"2024-05-03T20:34:35.597646+0000","last_peered":"2024-05-03T20:34:35.597646+0000","last_clean":"2024-05-03T20:34:35.597646+0000","last_became_active":"2024-05-03T20:33:40.709195+0000","last_became_peered":"2024-05-03T20:33:40.709195+0000","last_unstale":"2024-05-03T20:34:35.597646+0000","last_undegraded":"2024-05-03T20:34:35.597646+0000","last_fullsized":"2024-05-03T20:34:35.597646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:24:30.796765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.789390+0000","last_change":"2024-05-03T20:33:37.677729+0000","last_active":"2024-05-03T20:34:35.789390+0000","last_peered":"2024-05-03T20:34:35.789390+0000","last_clean":"2024-05-03T20:34:35.789390+0000","last_became_active":"2024-05-03T20:33:37.677285+0000","last_became_peered":"2024-05-03T20:33:37.677285+0000","last_unstale":"2024-05-03T20:34:35.789390+0000","last_undegraded":"2024-05-03T20:34:35.789390+0000","last_fullsized":"2024-05-03T20:34:35.789390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:01:46.289588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.594955+0000","last_change":"2024-05-03T20:33:40.708573+0000","last_active":"2024-05-03T20:34:35.594955+0000","last_peered":"2024-05-03T20:34:35.594955+0000","last_clean":"2024-05-03T20:34:35.594955+0000","last_became_active":"2024-05-03T20:33:40.708238+0000","last_became_peered":"2024-05-03T20:33:40.708238+0000","last_unstale":"2024-05-03T20:34:35.594955+0000","last_undegraded":"2024-05-03T20:34:35.594955+0000","last_fullsized":"2024-05-03T20:34:35.594955+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:21:06.975075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.593876+0000","last_change":"2024-05-03T20:33:37.683415+0000","last_active":"2024-05-03T20:34:35.593876+0000","last_peered":"2024-05-03T20:34:35.593876+0000","last_clean":"2024-05-03T20:34:35.593876+0000","last_became_active":"2024-05-03T20:33:37.682935+0000","last_became_peered":"2024-05-03T20:33:37.682935+0000","last_unstale":"2024-05-03T20:34:35.593876+0000","last_undegraded":"2024-05-03T20:34:35.593876+0000","last_fullsized":"2024-05-03T20:34:35.593876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:26:17.977777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.789573+0000","last_change":"2024-05-03T20:33:40.708116+0000","last_active":"2024-05-03T20:34:35.789573+0000","last_peered":"2024-05-03T20:34:35.789573+0000","last_clean":"2024-05-03T20:34:35.789573+0000","last_became_active":"2024-05-03T20:33:40.707625+0000","last_became_peered":"2024-05-03T20:33:40.707625+0000","last_unstale":"2024-05-03T20:34:35.789573+0000","last_undegraded":"2024-05-03T20:34:35.789573+0000","last_fullsized":"2024-05-03T20:34:35.789573+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:52:44.984252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598194+0000","last_change":"2024-05-03T20:33:37.685132+0000","last_active":"2024-05-03T20:34:35.598194+0000","last_peered":"2024-05-03T20:34:35.598194+0000","last_clean":"2024-05-03T20:34:35.598194+0000","last_became_active":"2024-05-03T20:33:37.684169+0000","last_became_peered":"2024-05-03T20:33:37.684169+0000","last_unstale":"2024-05-03T20:34:35.598194+0000","last_undegraded":"2024-05-03T20:34:35.598194+0000","last_fullsized":"2024-05-03T20:34:35.598194+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:41:24.163383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.595285+0000","last_change":"2024-05-03T20:33:40.713622+0000","last_active":"2024-05-03T20:34:35.595285+0000","last_peered":"2024-05-03T20:34:35.595285+0000","last_clean":"2024-05-03T20:34:35.595285+0000","last_became_active":"2024-05-03T20:33:40.712467+0000","last_became_peered":"2024-05-03T20:33:40.712467+0000","last_unstale":"2024-05-03T20:34:35.595285+0000","last_undegraded":"2024-05-03T20:34:35.595285+0000","last_fullsized":"2024-05-03T20:34:35.595285+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:40:07.948287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'5","reported_seq":76,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:35:30.686713+0000","last_change":"2024-05-03T20:34:35.604222+0000","last_active":"2024-05-03T20:35:30.686713+0000","last_peered":"2024-05-03T20:35:30.686713+0000","last_clean":"2024-05-03T20:35:30.686713+0000","last_became_active":"2024-05-03T20:34:35.597789+0000","last_became_peered":"2024-05-03T20:34:35.597789+0000","last_unstale":"2024-05-03T20:35:30.686713+0000","last_undegraded":"2024-05-03T20:35:30.686713+0000","last_fullsized":"2024-05-03T20:35:30.686713+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:43:39.527278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":57,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.596430+0000","last_change":"2024-05-03T20:33:40.710127+0000","last_active":"2024-05-03T20:34:35.596430+0000","last_peered":"2024-05-03T20:34:35.596430+0000","last_clean":"2024-05-03T20:34:35.596430+0000","last_became_active":"2024-05-03T20:33:40.709743+0000","last_became_peered":"2024-05-03T20:33:40.709743+0000","last_unstale":"2024-05-03T20:34:35.596430+0000","last_undegraded":"2024-05-03T20:34:35.596430+0000","last_fullsized":"2024-05-03T20:34:35.596430+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:01:48.638259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598938+0000","last_change":"2024-05-03T20:33:37.680009+0000","last_active":"2024-05-03T20:34:35.598938+0000","last_peered":"2024-05-03T20:34:35.598938+0000","last_clean":"2024-05-03T20:34:35.598938+0000","last_became_active":"2024-05-03T20:33:37.679577+0000","last_became_peered":"2024-05-03T20:33:37.679577+0000","last_unstale":"2024-05-03T20:34:35.598938+0000","last_undegraded":"2024-05-03T20:34:35.598938+0000","last_fullsized":"2024-05-03T20:34:35.598938+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:13:01.801473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.595992+0000","last_change":"2024-05-03T20:33:40.714713+0000","last_active":"2024-05-03T20:34:35.595992+0000","last_peered":"2024-05-03T20:34:35.595992+0000","last_clean":"2024-05-03T20:34:35.595992+0000","last_became_active":"2024-05-03T20:33:40.711743+0000","last_became_peered":"2024-05-03T20:33:40.711743+0000","last_unstale":"2024-05-03T20:34:35.595992+0000","last_undegraded":"2024-05-03T20:34:35.595992+0000","last_fullsized":"2024-05-03T20:34:35.595992+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:57:42.748953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.931542+0000","last_change":"2024-05-03T20:33:37.682038+0000","last_active":"2024-05-03T20:34:35.931542+0000","last_peered":"2024-05-03T20:34:35.931542+0000","last_clean":"2024-05-03T20:34:35.931542+0000","last_became_active":"2024-05-03T20:33:37.681559+0000","last_became_peered":"2024-05-03T20:33:37.681559+0000","last_unstale":"2024-05-03T20:34:35.931542+0000","last_undegraded":"2024-05-03T20:34:35.931542+0000","last_fullsized":"2024-05-03T20:34:35.931542+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:36.652267+0000","last_clean_scrub_stamp":"2024-05-03T20:33:36.652267+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:12:49.532764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.596370+0000","last_change":"2024-05-03T20:33:40.708025+0000","last_active":"2024-05-03T20:34:35.596370+0000","last_peered":"2024-05-03T20:34:35.596370+0000","last_clean":"2024-05-03T20:34:35.596370+0000","last_became_active":"2024-05-03T20:33:40.707601+0000","last_became_peered":"2024-05-03T20:33:40.707601+0000","last_unstale":"2024-05-03T20:34:35.596370+0000","last_undegraded":"2024-05-03T20:34:35.596370+0000","last_fullsized":"2024-05-03T20:34:35.596370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:34:32.060247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.595077+0000","last_change":"2024-05-03T20:33:40.711728+0000","last_active":"2024-05-03T20:34:35.595077+0000","last_peered":"2024-05-03T20:34:35.595077+0000","last_clean":"2024-05-03T20:34:35.595077+0000","last_became_active":"2024-05-03T20:33:40.711367+0000","last_became_peered":"2024-05-03T20:33:40.711367+0000","last_unstale":"2024-05-03T20:34:35.595077+0000","last_undegraded":"2024-05-03T20:34:35.595077+0000","last_fullsized":"2024-05-03T20:34:35.595077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:07:17.956339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.933014+0000","last_change":"2024-05-03T20:34:35.933014+0000","last_active":"2024-05-03T20:34:35.933014+0000","last_peered":"2024-05-03T20:34:35.933014+0000","last_clean":"2024-05-03T20:34:35.933014+0000","last_became_active":"2024-05-03T20:34:35.932407+0000","last_became_peered":"2024-05-03T20:34:35.932407+0000","last_unstale":"2024-05-03T20:34:35.933014+0000","last_undegraded":"2024-05-03T20:34:35.933014+0000","last_fullsized":"2024-05-03T20:34:35.933014+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:09:46.789627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.593591+0000","last_change":"2024-05-03T20:33:40.705468+0000","last_active":"2024-05-03T20:34:35.593591+0000","last_peered":"2024-05-03T20:34:35.593591+0000","last_clean":"2024-05-03T20:34:35.593591+0000","last_became_active":"2024-05-03T20:33:40.705114+0000","last_became_peered":"2024-05-03T20:33:40.705114+0000","last_unstale":"2024-05-03T20:34:35.593591+0000","last_undegraded":"2024-05-03T20:34:35.593591+0000","last_fullsized":"2024-05-03T20:34:35.593591+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:20:05.549638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.594111+0000","last_change":"2024-05-03T20:33:40.713282+0000","last_active":"2024-05-03T20:34:35.594111+0000","last_peered":"2024-05-03T20:34:35.594111+0000","last_clean":"2024-05-03T20:34:35.594111+0000","last_became_active":"2024-05-03T20:33:40.712674+0000","last_became_peered":"2024-05-03T20:33:40.712674+0000","last_unstale":"2024-05-03T20:34:35.594111+0000","last_undegraded":"2024-05-03T20:34:35.594111+0000","last_fullsized":"2024-05-03T20:34:35.594111+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:46:53.034439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.931576+0000","last_change":"2024-05-03T20:33:40.707870+0000","last_active":"2024-05-03T20:34:35.931576+0000","last_peered":"2024-05-03T20:34:35.931576+0000","last_clean":"2024-05-03T20:34:35.931576+0000","last_became_active":"2024-05-03T20:33:40.706849+0000","last_became_peered":"2024-05-03T20:33:40.706849+0000","last_unstale":"2024-05-03T20:34:35.931576+0000","last_undegraded":"2024-05-03T20:34:35.931576+0000","last_fullsized":"2024-05-03T20:34:35.931576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:23:18.030508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.596075+0000","last_change":"2024-05-03T20:33:40.716365+0000","last_active":"2024-05-03T20:34:35.596075+0000","last_peered":"2024-05-03T20:34:35.596075+0000","last_clean":"2024-05-03T20:34:35.596075+0000","last_became_active":"2024-05-03T20:33:40.716010+0000","last_became_peered":"2024-05-03T20:33:40.716010+0000","last_unstale":"2024-05-03T20:34:35.596075+0000","last_undegraded":"2024-05-03T20:34:35.596075+0000","last_fullsized":"2024-05-03T20:34:35.596075+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:29:08.459822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.594842+0000","last_change":"2024-05-03T20:33:40.712105+0000","last_active":"2024-05-03T20:34:35.594842+0000","last_peered":"2024-05-03T20:34:35.594842+0000","last_clean":"2024-05-03T20:34:35.594842+0000","last_became_active":"2024-05-03T20:33:40.711753+0000","last_became_peered":"2024-05-03T20:33:40.711753+0000","last_unstale":"2024-05-03T20:34:35.594842+0000","last_undegraded":"2024-05-03T20:34:35.594842+0000","last_fullsized":"2024-05-03T20:34:35.594842+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:29:14.895475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.789838+0000","last_change":"2024-05-03T20:33:40.713143+0000","last_active":"2024-05-03T20:34:35.789838+0000","last_peered":"2024-05-03T20:34:35.789838+0000","last_clean":"2024-05-03T20:34:35.789838+0000","last_became_active":"2024-05-03T20:33:40.712776+0000","last_became_peered":"2024-05-03T20:33:40.712776+0000","last_unstale":"2024-05-03T20:34:35.789838+0000","last_undegraded":"2024-05-03T20:34:35.789838+0000","last_fullsized":"2024-05-03T20:34:35.789838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:43:23.345103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598063+0000","last_change":"2024-05-03T20:33:40.706181+0000","last_active":"2024-05-03T20:34:35.598063+0000","last_peered":"2024-05-03T20:34:35.598063+0000","last_clean":"2024-05-03T20:34:35.598063+0000","last_became_active":"2024-05-03T20:33:40.705808+0000","last_became_peered":"2024-05-03T20:33:40.705808+0000","last_unstale":"2024-05-03T20:34:35.598063+0000","last_undegraded":"2024-05-03T20:34:35.598063+0000","last_fullsized":"2024-05-03T20:34:35.598063+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:26:00.035916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598274+0000","last_change":"2024-05-03T20:34:35.597903+0000","last_active":"2024-05-03T20:34:35.598274+0000","last_peered":"2024-05-03T20:34:35.598274+0000","last_clean":"2024-05-03T20:34:35.598274+0000","last_became_active":"2024-05-03T20:34:35.597570+0000","last_became_peered":"2024-05-03T20:34:35.597570+0000","last_unstale":"2024-05-03T20:34:35.598274+0000","last_undegraded":"2024-05-03T20:34:35.598274+0000","last_fullsized":"2024-05-03T20:34:35.598274+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:56:32.480374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.598307+0000","last_change":"2024-05-03T20:33:40.715949+0000","last_active":"2024-05-03T20:34:35.598307+0000","last_peered":"2024-05-03T20:34:35.598307+0000","last_clean":"2024-05-03T20:34:35.598307+0000","last_became_active":"2024-05-03T20:33:40.715366+0000","last_became_peered":"2024-05-03T20:33:40.715366+0000","last_unstale":"2024-05-03T20:34:35.598307+0000","last_undegraded":"2024-05-03T20:34:35.598307+0000","last_fullsized":"2024-05-03T20:34:35.598307+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:32:54.485990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.931437+0000","last_change":"2024-05-03T20:33:40.710447+0000","last_active":"2024-05-03T20:34:35.931437+0000","last_peered":"2024-05-03T20:34:35.931437+0000","last_clean":"2024-05-03T20:34:35.931437+0000","last_became_active":"2024-05-03T20:33:40.710107+0000","last_became_peered":"2024-05-03T20:33:40.710107+0000","last_unstale":"2024-05-03T20:34:35.931437+0000","last_undegraded":"2024-05-03T20:34:35.931437+0000","last_fullsized":"2024-05-03T20:34:35.931437+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:47:43.678947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-03T20:34:35.595647+0000","last_change":"2024-05-03T20:33:40.714081+0000","last_active":"2024-05-03T20:34:35.595647+0000","last_peered":"2024-05-03T20:34:35.595647+0000","last_clean":"2024-05-03T20:34:35.595647+0000","last_became_active":"2024-05-03T20:33:40.713678+0000","last_became_peered":"2024-05-03T20:33:40.713678+0000","last_unstale":"2024-05-03T20:34:35.595647+0000","last_undegraded":"2024-05-03T20:34:35.595647+0000","last_fullsized":"2024-05-03T20:34:35.595647+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T20:33:39.673042+0000","last_clean_scrub_stamp":"2024-05-03T20:33:39.673042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:24:30.128065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60760,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":941,"num_read_kb":556,"num_write":241,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1170,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":182280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612645,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":438259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808875,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1593344,"data_stored":1082844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972401,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":438189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168629,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":921600,"data_stored":438567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397561,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28508,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716644,"statfs":{"total":95995035648,"available":95965843456,"internally_reserved":0,"allocated":1519616,"data_stored":1031867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561087,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":438297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":962560,"data_stored":492375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":929792,"data_stored":439245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":974848,"data_stored":491425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117069,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":925696,"data_stored":437731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":439328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1531904,"data_stored":1030763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":55308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":54358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":55292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-03T20:35:36.185 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-03T20:35:36.185 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-03T20:35:36.185 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-03T20:35:36.185 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-03T20:35:36.426 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:35:36.788 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:36 smithi181 ceph-mon[29801]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T20:35:36.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:36 smithi196 ceph-mon[32858]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T20:35:36.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:36 smithi204 ceph-mon[33595]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T20:35:37.775 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:37 smithi181 ceph-mon[29801]: pgmap v344: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-03T20:35:37.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:37 smithi196 ceph-mon[32858]: pgmap v344: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-03T20:35:37.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:37 smithi204 ceph-mon[33595]: pgmap v344: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-03T20:35:38.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.145+0000 7fb532508700 1 -- 172.21.15.181:0/3713293990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb52c06bbf0 msgr2=0x7fb52c06c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:38.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.145+0000 7fb532508700 1 --2- 172.21.15.181:0/3713293990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb52c06bbf0 0x7fb52c06c070 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb51c009990 tx=0x7fb51c009ca0 comp rx=0 tx=0).stop 2024-05-03T20:35:38.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.145+0000 7fb532508700 1 -- 172.21.15.181:0/3713293990 shutdown_connections 2024-05-03T20:35:38.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.145+0000 7fb532508700 1 --2- 172.21.15.181:0/3713293990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb52c06c5b0 0x7fb52c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:38.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.145+0000 7fb532508700 1 --2- 172.21.15.181:0/3713293990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb52c109510 0x7fb52c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:38.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.145+0000 7fb532508700 1 --2- 172.21.15.181:0/3713293990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb52c06bbf0 0x7fb52c06c070 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:38.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.145+0000 7fb532508700 1 -- 172.21.15.181:0/3713293990 >> 172.21.15.181:0/3713293990 conn(0x7fb52c079370 msgr2=0x7fb52c079780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:38.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.145+0000 7fb532508700 1 -- 172.21.15.181:0/3713293990 shutdown_connections 2024-05-03T20:35:38.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.145+0000 7fb532508700 1 -- 172.21.15.181:0/3713293990 wait complete. 2024-05-03T20:35:38.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.146+0000 7fb532508700 1 Processor -- start 2024-05-03T20:35:38.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.146+0000 7fb532508700 1 -- start start 2024-05-03T20:35:38.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.146+0000 7fb532508700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb52c06bbf0 0x7fb52c19f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:38.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.146+0000 7fb532508700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb52c06c5b0 0x7fb52c19fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.146+0000 7fb532508700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb52c109510 0x7fb52c199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.146+0000 7fb532508700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c113720 con 0x7fb52c109510 2024-05-03T20:35:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.146+0000 7fb532508700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1135a0 con 0x7fb52c06c5b0 2024-05-03T20:35:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.146+0000 7fb532508700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1138a0 con 0x7fb52c06bbf0 2024-05-03T20:35:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.146+0000 7fb530aa5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb52c109510 0x7fb52c199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.147+0000 7fb530aa5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb52c109510 0x7fb52c199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54712/0 (socket says 172.21.15.181:54712) 2024-05-03T20:35:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.147+0000 7fb530aa5700 1 -- 172.21.15.181:0/2545194523 learned_addr learned my addr 172.21.15.181:0/2545194523 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.147+0000 7fb52bfff700 1 --2- 172.21.15.181:0/2545194523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb52c06bbf0 0x7fb52c19f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.147+0000 7fb52b7fe700 1 --2- 172.21.15.181:0/2545194523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb52c06c5b0 0x7fb52c19fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.147+0000 7fb530aa5700 1 -- 172.21.15.181:0/2545194523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb52c06bbf0 msgr2=0x7fb52c19f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.147+0000 7fb530aa5700 1 --2- 172.21.15.181:0/2545194523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb52c06bbf0 0x7fb52c19f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.147+0000 7fb530aa5700 1 -- 172.21.15.181:0/2545194523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb52c06c5b0 msgr2=0x7fb52c19fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:38.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.147+0000 7fb530aa5700 1 --2- 172.21.15.181:0/2545194523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb52c06c5b0 0x7fb52c19fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:38.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.147+0000 7fb530aa5700 1 -- 172.21.15.181:0/2545194523 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb52c19a240 con 0x7fb52c109510 2024-05-03T20:35:38.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.147+0000 7fb52bfff700 1 --2- 172.21.15.181:0/2545194523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb52c06bbf0 0x7fb52c19f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:35:38.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.147+0000 7fb52b7fe700 1 --2- 172.21.15.181:0/2545194523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb52c06c5b0 0x7fb52c19fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:35:38.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.147+0000 7fb530aa5700 1 --2- 172.21.15.181:0/2545194523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb52c109510 0x7fb52c199ae0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb52400b970 tx=0x7fb52400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:38.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.147+0000 7fb5297fa700 1 -- 172.21.15.181:0/2545194523 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb524019070 con 0x7fb52c109510 2024-05-03T20:35:38.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.147+0000 7fb5297fa700 1 -- 172.21.15.181:0/2545194523 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb524014440 con 0x7fb52c109510 2024-05-03T20:35:38.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.147+0000 7fb532508700 1 -- 172.21.15.181:0/2545194523 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb52c19a530 con 0x7fb52c109510 2024-05-03T20:35:38.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.147+0000 7fb5297fa700 1 -- 172.21.15.181:0/2545194523 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52400ce60 con 0x7fb52c109510 2024-05-03T20:35:38.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.147+0000 7fb532508700 1 -- 172.21.15.181:0/2545194523 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb52c1a67f0 con 0x7fb52c109510 2024-05-03T20:35:38.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.148+0000 7fb532508700 1 -- 172.21.15.181:0/2545194523 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb52c10b9f0 con 0x7fb52c109510 2024-05-03T20:35:38.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.149+0000 7fb5297fa700 1 -- 172.21.15.181:0/2545194523 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fb52401c070 con 0x7fb52c109510 2024-05-03T20:35:38.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.150+0000 7fb5297fa700 1 --2- 172.21.15.181:0/2545194523 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb5140b2990 0x7fb5140b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:38.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.150+0000 7fb52bfff700 1 --2- 172.21.15.181:0/2545194523 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb5140b2990 0x7fb5140b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:38.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.150+0000 7fb5297fa700 1 -- 172.21.15.181:0/2545194523 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7fb5240e1b00 con 0x7fb52c109510 2024-05-03T20:35:38.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.151+0000 7fb52bfff700 1 --2- 172.21.15.181:0/2545194523 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb5140b2990 0x7fb5140b4e50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb5200099d0 tx=0x7fb520008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:38.153 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.154+0000 7fb5297fa700 1 -- 172.21.15.181:0/2545194523 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb5240a97d0 con 0x7fb52c109510 2024-05-03T20:35:38.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.403+0000 7fb532508700 1 -- 172.21.15.181:0/2545194523 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb52c19b000 con 0x7fb52c109510 2024-05-03T20:35:38.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.403+0000 7fb5297fa700 1 -- 172.21.15.181:0/2545194523 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fb52c19b000 con 0x7fb52c109510 2024-05-03T20:35:38.403 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:35:38.403 INFO:teuthology.orchestra.run.smithi181.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-03T20:35:38.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.405+0000 7fb532508700 1 -- 172.21.15.181:0/2545194523 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb5140b2990 msgr2=0x7fb5140b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:38.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.405+0000 7fb532508700 1 --2- 172.21.15.181:0/2545194523 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb5140b2990 0x7fb5140b4e50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb5200099d0 tx=0x7fb520008040 comp rx=0 tx=0).stop 2024-05-03T20:35:38.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.405+0000 7fb532508700 1 -- 172.21.15.181:0/2545194523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb52c109510 msgr2=0x7fb52c199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:38.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.405+0000 7fb532508700 1 --2- 172.21.15.181:0/2545194523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb52c109510 0x7fb52c199ae0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb52400b970 tx=0x7fb52400bd30 comp rx=0 tx=0).stop 2024-05-03T20:35:38.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.405+0000 7fb532508700 1 -- 172.21.15.181:0/2545194523 shutdown_connections 2024-05-03T20:35:38.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.405+0000 7fb532508700 1 --2- 172.21.15.181:0/2545194523 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb5140b2990 0x7fb5140b4e50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:38.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.405+0000 7fb532508700 1 --2- 172.21.15.181:0/2545194523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb52c109510 0x7fb52c199ae0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:38.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.405+0000 7fb532508700 1 --2- 172.21.15.181:0/2545194523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb52c06bbf0 0x7fb52c19f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:38.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.405+0000 7fb532508700 1 --2- 172.21.15.181:0/2545194523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb52c06c5b0 0x7fb52c19fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:38.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.405+0000 7fb532508700 1 -- 172.21.15.181:0/2545194523 >> 172.21.15.181:0/2545194523 conn(0x7fb52c079370 msgr2=0x7fb52c101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:38.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.405+0000 7fb532508700 1 -- 172.21.15.181:0/2545194523 shutdown_connections 2024-05-03T20:35:38.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:38.405+0000 7fb532508700 1 -- 172.21.15.181:0/2545194523 wait complete. 2024-05-03T20:35:38.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:38 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/2545194523' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-03T20:35:38.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:38 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/2545194523' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-03T20:35:38.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:38 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/2545194523' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-03T20:35:39.069 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-03T20:35:39.070 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T20:35:39.308 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:35:39.711 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:39 smithi181 ceph-mon[29801]: pgmap v345: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-03T20:35:39.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:39 smithi196 ceph-mon[32858]: pgmap v345: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-03T20:35:39.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:39 smithi204 ceph-mon[33595]: pgmap v345: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-03T20:35:40.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.837+0000 7f3247c08700 1 -- 172.21.15.181:0/1916039590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3240109510 msgr2=0x7f32401098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:40.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.837+0000 7f3247c08700 1 --2- 172.21.15.181:0/1916039590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3240109510 0x7f32401098f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f323c009990 tx=0x7f323c009ca0 comp rx=0 tx=0).stop 2024-05-03T20:35:40.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.837+0000 7f3247c08700 1 -- 172.21.15.181:0/1916039590 shutdown_connections 2024-05-03T20:35:40.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.837+0000 7f3247c08700 1 --2- 172.21.15.181:0/1916039590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f324006c590 0x7f3240110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:40.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.837+0000 7f3247c08700 1 --2- 172.21.15.181:0/1916039590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3240109510 0x7f32401098f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:40.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.837+0000 7f3247c08700 1 --2- 172.21.15.181:0/1916039590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f324006bbd0 0x7f324006c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:40.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.837+0000 7f3247c08700 1 -- 172.21.15.181:0/1916039590 >> 172.21.15.181:0/1916039590 conn(0x7f3240079350 msgr2=0x7f3240079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:40.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.838+0000 7f3247c08700 1 -- 172.21.15.181:0/1916039590 shutdown_connections 2024-05-03T20:35:40.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.838+0000 7f3247c08700 1 -- 172.21.15.181:0/1916039590 wait complete. 2024-05-03T20:35:40.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.838+0000 7f3247c08700 1 Processor -- start 2024-05-03T20:35:40.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.838+0000 7f3247c08700 1 -- start start 2024-05-03T20:35:40.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.838+0000 7f3247c08700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f324006bbd0 0x7f324019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:40.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.838+0000 7f3247c08700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f324006c590 0x7f324019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:40.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.838+0000 7f3247c08700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3240109510 0x7f32401998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:40.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.838+0000 7f3247c08700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3240113670 con 0x7f3240109510 2024-05-03T20:35:40.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.838+0000 7f3247c08700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f32401134f0 con 0x7f324006c590 2024-05-03T20:35:40.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.838+0000 7f3247c08700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f32401137f0 con 0x7f324006bbd0 2024-05-03T20:35:40.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.839+0000 7f32461a5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3240109510 0x7f32401998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:40.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.839+0000 7f32461a5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3240109510 0x7f32401998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54724/0 (socket says 172.21.15.181:54724) 2024-05-03T20:35:40.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.839+0000 7f32461a5700 1 -- 172.21.15.181:0/2495193150 learned_addr learned my addr 172.21.15.181:0/2495193150 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:40.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.839+0000 7f32459a4700 1 --2- 172.21.15.181:0/2495193150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f324006bbd0 0x7f324019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:40.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.839+0000 7f32451a3700 1 --2- 172.21.15.181:0/2495193150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f324006c590 0x7f324019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:40.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.839+0000 7f32461a5700 1 -- 172.21.15.181:0/2495193150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f324006bbd0 msgr2=0x7f324019f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:40.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.839+0000 7f32461a5700 1 --2- 172.21.15.181:0/2495193150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f324006bbd0 0x7f324019f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:40.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.839+0000 7f32461a5700 1 -- 172.21.15.181:0/2495193150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f324006c590 msgr2=0x7f324019fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:40.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.839+0000 7f32461a5700 1 --2- 172.21.15.181:0/2495193150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f324006c590 0x7f324019fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:40.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.839+0000 7f32461a5700 1 -- 172.21.15.181:0/2495193150 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324019a050 con 0x7f3240109510 2024-05-03T20:35:40.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.839+0000 7f32459a4700 1 --2- 172.21.15.181:0/2495193150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f324006bbd0 0x7f324019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:35:40.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.839+0000 7f32461a5700 1 --2- 172.21.15.181:0/2495193150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3240109510 0x7f32401998f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3238007ee0 tx=0x7f323800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:40.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.839+0000 7f3236ffd700 1 -- 172.21.15.181:0/2495193150 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323800dbd0 con 0x7f3240109510 2024-05-03T20:35:40.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.839+0000 7f3236ffd700 1 -- 172.21.15.181:0/2495193150 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f323800dd70 con 0x7f3240109510 2024-05-03T20:35:40.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.839+0000 7f3247c08700 1 -- 172.21.15.181:0/2495193150 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f324019a340 con 0x7f3240109510 2024-05-03T20:35:40.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.839+0000 7f3236ffd700 1 -- 172.21.15.181:0/2495193150 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3238005990 con 0x7f3240109510 2024-05-03T20:35:40.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.839+0000 7f3247c08700 1 -- 172.21.15.181:0/2495193150 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32401a6600 con 0x7f3240109510 2024-05-03T20:35:40.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.840+0000 7f3247c08700 1 -- 172.21.15.181:0/2495193150 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f324010b9f0 con 0x7f3240109510 2024-05-03T20:35:40.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.841+0000 7f3236ffd700 1 -- 172.21.15.181:0/2495193150 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f323801a020 con 0x7f3240109510 2024-05-03T20:35:40.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.842+0000 7f3236ffd700 1 --2- 172.21.15.181:0/2495193150 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f322c0b2910 0x7f322c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:40.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.842+0000 7f32459a4700 1 --2- 172.21.15.181:0/2495193150 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f322c0b2910 0x7f322c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:40.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.842+0000 7f3236ffd700 1 -- 172.21.15.181:0/2495193150 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f32380e2210 con 0x7f3240109510 2024-05-03T20:35:40.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.842+0000 7f32459a4700 1 --2- 172.21.15.181:0/2495193150 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f322c0b2910 0x7f322c0b4dd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f323c006010 tx=0x7f323c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:40.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:40.846+0000 7f3236ffd700 1 -- 172.21.15.181:0/2495193150 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3238010070 con 0x7f3240109510 2024-05-03T20:35:41.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:41.094+0000 7f3247c08700 1 -- 172.21.15.181:0/2495193150 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f324019ae20 con 0x7f3240109510 2024-05-03T20:35:41.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:41.096+0000 7f3236ffd700 1 -- 172.21.15.181:0/2495193150 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11954 (secure 0 0 0) 0x7f32380a9dd0 con 0x7f3240109510 2024-05-03T20:35:41.095 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:35:41.095 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2414172454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2414172454},{"type":"v1","addr":"172.21.15.181:6835","nonce":2414172454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/204127189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":204127189},{"type":"v1","addr":"172.21.15.181:6837","nonce":204127189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T20:33:42.487965+0000","modified":"2024-05-03T20:34:00.359735+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24475,"mds_2":24464,"mds_3":24463,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14562":{"gid":14562,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6839/257361276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":257361276},{"type":"v1","addr":"172.21.15.181:6839","nonce":257361276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6841/2403160115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2403160115},{"type":"v1","addr":"172.21.15.181:6841","nonce":2403160115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3975688022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3975688022},{"type":"v1","addr":"172.21.15.196:6833","nonce":3975688022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2409628823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2409628823},{"type":"v1","addr":"172.21.15.204:6833","nonce":2409628823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1820206106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1820206106},{"type":"v1","addr":"172.21.15.196:6835","nonce":1820206106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6835/3870850287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3870850287},{"type":"v1","addr":"172.21.15.204:6835","nonce":3870850287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/3006728559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3006728559},{"type":"v1","addr":"172.21.15.204:6837","nonce":3006728559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6837/1529420259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1529420259},{"type":"v1","addr":"172.21.15.196:6837","nonce":1529420259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/1644828559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1644828559},{"type":"v1","addr":"172.21.15.204:6839","nonce":1644828559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.196:6839/1182826743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1182826743},{"type":"v1","addr":"172.21.15.196:6839","nonce":1182826743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24457,24464,24463,24475,24488]},"id":1}]} 2024-05-03T20:35:41.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:41.098+0000 7f3247c08700 1 -- 172.21.15.181:0/2495193150 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f322c0b2910 msgr2=0x7f322c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:41.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:41.098+0000 7f3247c08700 1 --2- 172.21.15.181:0/2495193150 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f322c0b2910 0x7f322c0b4dd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f323c006010 tx=0x7f323c01a040 comp rx=0 tx=0).stop 2024-05-03T20:35:41.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:41.098+0000 7f3247c08700 1 -- 172.21.15.181:0/2495193150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3240109510 msgr2=0x7f32401998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:41.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:41.098+0000 7f3247c08700 1 --2- 172.21.15.181:0/2495193150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3240109510 0x7f32401998f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3238007ee0 tx=0x7f323800d3b0 comp rx=0 tx=0).stop 2024-05-03T20:35:41.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:41.098+0000 7f3247c08700 1 -- 172.21.15.181:0/2495193150 shutdown_connections 2024-05-03T20:35:41.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:41.098+0000 7f3247c08700 1 --2- 172.21.15.181:0/2495193150 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f322c0b2910 0x7f322c0b4dd0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:41.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:41.098+0000 7f3247c08700 1 --2- 172.21.15.181:0/2495193150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3240109510 0x7f32401998f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:41.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:41.098+0000 7f3247c08700 1 --2- 172.21.15.181:0/2495193150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f324006bbd0 0x7f324019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:41.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:41.098+0000 7f3247c08700 1 --2- 172.21.15.181:0/2495193150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f324006c590 0x7f324019fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:41.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:41.098+0000 7f3247c08700 1 -- 172.21.15.181:0/2495193150 >> 172.21.15.181:0/2495193150 conn(0x7f3240079350 msgr2=0x7f3240101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:41.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:41.098+0000 7f3247c08700 1 -- 172.21.15.181:0/2495193150 shutdown_connections 2024-05-03T20:35:41.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:41.098+0000 7f3247c08700 1 -- 172.21.15.181:0/2495193150 wait complete. 2024-05-03T20:35:41.099 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 27 2024-05-03T20:35:41.673 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:41 smithi181 ceph-mon[29801]: pgmap v346: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-03T20:35:41.673 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:41 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/2495193150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:35:41.701 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T20:35:41.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:41 smithi196 ceph-mon[32858]: pgmap v346: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-03T20:35:41.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:41 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/2495193150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:35:41.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:41 smithi204 ceph-mon[33595]: pgmap v346: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-03T20:35:41.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:41 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/2495193150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:35:41.940 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:35:43.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.560+0000 7f63f89fe700 1 -- 172.21.15.181:0/177336364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63f4105690 msgr2=0x7f63f4105b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:43.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.560+0000 7f63f89fe700 1 --2- 172.21.15.181:0/177336364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63f4105690 0x7f63f4105b10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009990 tx=0x7f63e8009ca0 comp rx=0 tx=0).stop 2024-05-03T20:35:43.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.561+0000 7f63f89fe700 1 -- 172.21.15.181:0/177336364 shutdown_connections 2024-05-03T20:35:43.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.561+0000 7f63f89fe700 1 --2- 172.21.15.181:0/177336364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63f4106050 0x7f63f4112560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:43.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.561+0000 7f63f89fe700 1 --2- 172.21.15.181:0/177336364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63f410b690 0x7f63f410ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:43.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.561+0000 7f63f89fe700 1 --2- 172.21.15.181:0/177336364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63f4105690 0x7f63f4105b10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:43.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.561+0000 7f63f89fe700 1 -- 172.21.15.181:0/177336364 >> 172.21.15.181:0/177336364 conn(0x7f63f4101180 msgr2=0x7f63f41035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:43.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.561+0000 7f63f89fe700 1 -- 172.21.15.181:0/177336364 shutdown_connections 2024-05-03T20:35:43.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.561+0000 7f63f89fe700 1 -- 172.21.15.181:0/177336364 wait complete. 2024-05-03T20:35:43.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.561+0000 7f63f89fe700 1 Processor -- start 2024-05-03T20:35:43.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.561+0000 7f63f89fe700 1 -- start start 2024-05-03T20:35:43.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.562+0000 7f63f89fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63f4105690 0x7f63f419bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:43.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.562+0000 7f63f89fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63f4106050 0x7f63f419c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:43.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.562+0000 7f63f89fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63f410b690 0x7f63f41a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:43.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.562+0000 7f63f89fe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f63f41158d0 con 0x7f63f4105690 2024-05-03T20:35:43.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.562+0000 7f63f89fe700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f63f4115750 con 0x7f63f4106050 2024-05-03T20:35:43.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.562+0000 7f63f89fe700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f63f4115a50 con 0x7f63f410b690 2024-05-03T20:35:43.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.562+0000 7f63f259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63f4105690 0x7f63f419bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:43.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.562+0000 7f63f259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63f4105690 0x7f63f419bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54738/0 (socket says 172.21.15.181:54738) 2024-05-03T20:35:43.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.562+0000 7f63f259c700 1 -- 172.21.15.181:0/4178799270 learned_addr learned my addr 172.21.15.181:0/4178799270 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:43.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.562+0000 7f63f2d9d700 1 --2- 172.21.15.181:0/4178799270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63f410b690 0x7f63f41a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:43.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.562+0000 7f63f259c700 1 -- 172.21.15.181:0/4178799270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63f410b690 msgr2=0x7f63f41a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:43.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.562+0000 7f63f1d9b700 1 --2- 172.21.15.181:0/4178799270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63f4106050 0x7f63f419c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:43.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.562+0000 7f63f259c700 1 --2- 172.21.15.181:0/4178799270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63f410b690 0x7f63f41a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:43.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.562+0000 7f63f259c700 1 -- 172.21.15.181:0/4178799270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63f4106050 msgr2=0x7f63f419c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:43.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.562+0000 7f63f259c700 1 --2- 172.21.15.181:0/4178799270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63f4106050 0x7f63f419c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:43.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.562+0000 7f63f259c700 1 -- 172.21.15.181:0/4178799270 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f41a0db0 con 0x7f63f4105690 2024-05-03T20:35:43.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.562+0000 7f63f2d9d700 1 --2- 172.21.15.181:0/4178799270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63f410b690 0x7f63f41a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:35:43.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.562+0000 7f63f259c700 1 --2- 172.21.15.181:0/4178799270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63f4105690 0x7f63f419bda0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f63e400c920 tx=0x7f63e400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:43.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.562+0000 7f63e37fe700 1 -- 172.21.15.181:0/4178799270 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e4007b00 con 0x7f63f4105690 2024-05-03T20:35:43.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.562+0000 7f63e37fe700 1 -- 172.21.15.181:0/4178799270 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63e4004ce0 con 0x7f63f4105690 2024-05-03T20:35:43.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.562+0000 7f63f89fe700 1 -- 172.21.15.181:0/4178799270 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63f41a10a0 con 0x7f63f4105690 2024-05-03T20:35:43.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.562+0000 7f63e37fe700 1 -- 172.21.15.181:0/4178799270 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e4005650 con 0x7f63f4105690 2024-05-03T20:35:43.564 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.563+0000 7f63f89fe700 1 -- 172.21.15.181:0/4178799270 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63f41a8920 con 0x7f63f4105690 2024-05-03T20:35:43.565 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.563+0000 7f63f89fe700 1 -- 172.21.15.181:0/4178799270 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63f406a190 con 0x7f63f4105690 2024-05-03T20:35:43.565 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:43 smithi181 ceph-mon[29801]: pgmap v347: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-03T20:35:43.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.565+0000 7f63e37fe700 1 -- 172.21.15.181:0/4178799270 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f63e4004750 con 0x7f63f4105690 2024-05-03T20:35:43.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.566+0000 7f63e37fe700 1 --2- 172.21.15.181:0/4178799270 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f63dc0b2910 0x7f63dc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:43.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.566+0000 7f63f1d9b700 1 --2- 172.21.15.181:0/4178799270 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f63dc0b2910 0x7f63dc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:43.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.566+0000 7f63e37fe700 1 -- 172.21.15.181:0/4178799270 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f63e40e23f0 con 0x7f63f4105690 2024-05-03T20:35:43.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.567+0000 7f63f1d9b700 1 --2- 172.21.15.181:0/4178799270 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f63dc0b2910 0x7f63dc0b4dd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f63e8005cd0 tx=0x7f63e8005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:43.569 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.570+0000 7f63e37fe700 1 -- 172.21.15.181:0/4178799270 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f63e4014030 con 0x7f63f4105690 2024-05-03T20:35:43.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.821+0000 7f63f89fe700 1 -- 172.21.15.181:0/4178799270 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63f402e510 con 0x7f63f4105690 2024-05-03T20:35:43.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.822+0000 7f63e37fe700 1 -- 172.21.15.181:0/4178799270 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11954 (secure 0 0 0) 0x7f63e40a9fb0 con 0x7f63f4105690 2024-05-03T20:35:43.821 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:35:43.821 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2414172454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2414172454},{"type":"v1","addr":"172.21.15.181:6835","nonce":2414172454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/204127189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":204127189},{"type":"v1","addr":"172.21.15.181:6837","nonce":204127189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T20:33:42.487965+0000","modified":"2024-05-03T20:34:00.359735+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24475,"mds_2":24464,"mds_3":24463,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14562":{"gid":14562,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6839/257361276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":257361276},{"type":"v1","addr":"172.21.15.181:6839","nonce":257361276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6841/2403160115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2403160115},{"type":"v1","addr":"172.21.15.181:6841","nonce":2403160115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3975688022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3975688022},{"type":"v1","addr":"172.21.15.196:6833","nonce":3975688022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2409628823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2409628823},{"type":"v1","addr":"172.21.15.204:6833","nonce":2409628823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1820206106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1820206106},{"type":"v1","addr":"172.21.15.196:6835","nonce":1820206106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6835/3870850287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3870850287},{"type":"v1","addr":"172.21.15.204:6835","nonce":3870850287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/3006728559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3006728559},{"type":"v1","addr":"172.21.15.204:6837","nonce":3006728559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6837/1529420259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1529420259},{"type":"v1","addr":"172.21.15.196:6837","nonce":1529420259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/1644828559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1644828559},{"type":"v1","addr":"172.21.15.204:6839","nonce":1644828559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.196:6839/1182826743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1182826743},{"type":"v1","addr":"172.21.15.196:6839","nonce":1182826743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24457,24464,24463,24475,24488]},"id":1}]} 2024-05-03T20:35:43.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.824+0000 7f63f89fe700 1 -- 172.21.15.181:0/4178799270 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f63dc0b2910 msgr2=0x7f63dc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:43.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.824+0000 7f63f89fe700 1 --2- 172.21.15.181:0/4178799270 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f63dc0b2910 0x7f63dc0b4dd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f63e8005cd0 tx=0x7f63e8005c40 comp rx=0 tx=0).stop 2024-05-03T20:35:43.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.824+0000 7f63f89fe700 1 -- 172.21.15.181:0/4178799270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63f4105690 msgr2=0x7f63f419bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:43.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.824+0000 7f63f89fe700 1 --2- 172.21.15.181:0/4178799270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63f4105690 0x7f63f419bda0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f63e400c920 tx=0x7f63e400cce0 comp rx=0 tx=0).stop 2024-05-03T20:35:43.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.824+0000 7f63f89fe700 1 -- 172.21.15.181:0/4178799270 shutdown_connections 2024-05-03T20:35:43.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.824+0000 7f63f89fe700 1 --2- 172.21.15.181:0/4178799270 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f63dc0b2910 0x7f63dc0b4dd0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:43.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.824+0000 7f63f89fe700 1 --2- 172.21.15.181:0/4178799270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63f410b690 0x7f63f41a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:43.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.824+0000 7f63f89fe700 1 --2- 172.21.15.181:0/4178799270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63f4105690 0x7f63f419bda0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:43.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.824+0000 7f63f89fe700 1 --2- 172.21.15.181:0/4178799270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63f4106050 0x7f63f419c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:43.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.824+0000 7f63f89fe700 1 -- 172.21.15.181:0/4178799270 >> 172.21.15.181:0/4178799270 conn(0x7f63f4101180 msgr2=0x7f63f4101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:43.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.825+0000 7f63f89fe700 1 -- 172.21.15.181:0/4178799270 shutdown_connections 2024-05-03T20:35:43.824 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:43.825+0000 7f63f89fe700 1 -- 172.21.15.181:0/4178799270 wait complete. 2024-05-03T20:35:43.824 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 27 2024-05-03T20:35:43.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:43 smithi196 ceph-mon[32858]: pgmap v347: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-03T20:35:43.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:43 smithi204 ceph-mon[33595]: pgmap v347: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-03T20:35:44.389 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T20:35:44.638 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:35:44.673 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:44 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/4178799270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:35:44.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:44 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/4178799270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:35:44.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:44 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/4178799270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:35:45.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:45 smithi181 ceph-mon[29801]: pgmap v348: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2024-05-03T20:35:45.757 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:45 smithi196 ceph-mon[32858]: pgmap v348: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2024-05-03T20:35:45.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:45 smithi204 ceph-mon[33595]: pgmap v348: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2024-05-03T20:35:46.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.309+0000 7f5c26ebe700 1 -- 172.21.15.181:0/998038333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c20108410 msgr2=0x7f5c201087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:46.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.309+0000 7f5c26ebe700 1 --2- 172.21.15.181:0/998038333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c20108410 0x7f5c201087f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5c14009990 tx=0x7f5c14009ca0 comp rx=0 tx=0).stop 2024-05-03T20:35:46.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.309+0000 7f5c26ebe700 1 -- 172.21.15.181:0/998038333 shutdown_connections 2024-05-03T20:35:46.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.309+0000 7f5c26ebe700 1 --2- 172.21.15.181:0/998038333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c20102d60 0x7f5c2010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:46.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.309+0000 7f5c26ebe700 1 --2- 172.21.15.181:0/998038333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c20108410 0x7f5c201087f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:46.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.309+0000 7f5c26ebe700 1 --2- 172.21.15.181:0/998038333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c201023a0 0x7f5c20102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:46.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.309+0000 7f5c26ebe700 1 -- 172.21.15.181:0/998038333 >> 172.21.15.181:0/998038333 conn(0x7f5c20079350 msgr2=0x7f5c20079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:46.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.309+0000 7f5c26ebe700 1 -- 172.21.15.181:0/998038333 shutdown_connections 2024-05-03T20:35:46.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.309+0000 7f5c26ebe700 1 -- 172.21.15.181:0/998038333 wait complete. 2024-05-03T20:35:46.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.310+0000 7f5c26ebe700 1 Processor -- start 2024-05-03T20:35:46.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.310+0000 7f5c26ebe700 1 -- start start 2024-05-03T20:35:46.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.310+0000 7f5c26ebe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c201023a0 0x7f5c2019beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:46.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.310+0000 7f5c26ebe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c20102d60 0x7f5c2019c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:46.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.310+0000 7f5c26ebe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c20108410 0x7f5c201a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:46.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.310+0000 7f5c26ebe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5c20113730 con 0x7f5c201023a0 2024-05-03T20:35:46.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.310+0000 7f5c26ebe700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5c201135b0 con 0x7f5c20108410 2024-05-03T20:35:46.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.310+0000 7f5c26ebe700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5c201138b0 con 0x7f5c20102d60 2024-05-03T20:35:46.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.310+0000 7f5c24c5a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c201023a0 0x7f5c2019beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:46.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.310+0000 7f5c24c5a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c201023a0 0x7f5c2019beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42634/0 (socket says 172.21.15.181:42634) 2024-05-03T20:35:46.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.310+0000 7f5c24c5a700 1 -- 172.21.15.181:0/3416612891 learned_addr learned my addr 172.21.15.181:0/3416612891 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:46.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.310+0000 7f5c2545b700 1 --2- 172.21.15.181:0/3416612891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c20108410 0x7f5c201a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:46.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.310+0000 7f5c1ffff700 1 --2- 172.21.15.181:0/3416612891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c20102d60 0x7f5c2019c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:46.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.311+0000 7f5c24c5a700 1 -- 172.21.15.181:0/3416612891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c20102d60 msgr2=0x7f5c2019c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:46.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.311+0000 7f5c24c5a700 1 --2- 172.21.15.181:0/3416612891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c20102d60 0x7f5c2019c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:46.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.311+0000 7f5c24c5a700 1 -- 172.21.15.181:0/3416612891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c20108410 msgr2=0x7f5c201a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:46.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.311+0000 7f5c24c5a700 1 --2- 172.21.15.181:0/3416612891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c20108410 0x7f5c201a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:46.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.311+0000 7f5c24c5a700 1 -- 172.21.15.181:0/3416612891 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c201a0ec0 con 0x7f5c201023a0 2024-05-03T20:35:46.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.311+0000 7f5c24c5a700 1 --2- 172.21.15.181:0/3416612891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c201023a0 0x7f5c2019beb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5c140057f0 tx=0x7f5c14003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:46.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.311+0000 7f5c1dffb700 1 -- 172.21.15.181:0/3416612891 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c14027070 con 0x7f5c201023a0 2024-05-03T20:35:46.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.311+0000 7f5c26ebe700 1 -- 172.21.15.181:0/3416612891 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c201a1150 con 0x7f5c201023a0 2024-05-03T20:35:46.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.311+0000 7f5c1dffb700 1 -- 172.21.15.181:0/3416612891 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c140040a0 con 0x7f5c201023a0 2024-05-03T20:35:46.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.311+0000 7f5c1dffb700 1 -- 172.21.15.181:0/3416612891 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c14022dc0 con 0x7f5c201023a0 2024-05-03T20:35:46.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.311+0000 7f5c26ebe700 1 -- 172.21.15.181:0/3416612891 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c201a8a30 con 0x7f5c201023a0 2024-05-03T20:35:46.313 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.312+0000 7f5c26ebe700 1 -- 172.21.15.181:0/3416612891 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c2006a190 con 0x7f5c201023a0 2024-05-03T20:35:46.314 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.313+0000 7f5c1dffb700 1 -- 172.21.15.181:0/3416612891 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f5c1402a070 con 0x7f5c201023a0 2024-05-03T20:35:46.314 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.314+0000 7f5c1dffb700 1 --2- 172.21.15.181:0/3416612891 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5c100b2910 0x7f5c100b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:46.314 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.314+0000 7f5c1ffff700 1 --2- 172.21.15.181:0/3416612891 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5c100b2910 0x7f5c100b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:46.314 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.314+0000 7f5c1dffb700 1 -- 172.21.15.181:0/3416612891 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f5c140e3880 con 0x7f5c201023a0 2024-05-03T20:35:46.314 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.314+0000 7f5c1ffff700 1 --2- 172.21.15.181:0/3416612891 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5c100b2910 0x7f5c100b4dd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c006fd0 tx=0x7f5c0c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:46.316 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.317+0000 7f5c1dffb700 1 -- 172.21.15.181:0/3416612891 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5c14031020 con 0x7f5c201023a0 2024-05-03T20:35:46.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.530+0000 7f5c26ebe700 1 -- 172.21.15.181:0/3416612891 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c2002e510 con 0x7f5c201023a0 2024-05-03T20:35:46.530 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.531+0000 7f5c1dffb700 1 -- 172.21.15.181:0/3416612891 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22710 (secure 0 0 0) 0x7f5c14013070 con 0x7f5c201023a0 2024-05-03T20:35:46.530 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:35:46.530 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":83,"fsid":"1cd058ee-098b-11ef-bc97-c7b262605968","created":"2024-05-03T20:25:39.427815+0000","modified":"2024-05-03T20:34:35.581238+0000","last_up_change":"2024-05-03T20:32:38.854730+0000","last_in_change":"2024-05-03T20:32:18.787388+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T20:29:01.855187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T20:33:35.737273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T20:33:39.337556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5f594d97-10a4-4f77-aebc-c00dbe92a4af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6803","nonce":1683973278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6805","nonce":1683973278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6809","nonce":1683973278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6807","nonce":1683973278}]},"public_addr":"172.21.15.181:6803/1683973278","cluster_addr":"172.21.15.181:6805/1683973278","heartbeat_back_addr":"172.21.15.181:6809/1683973278","heartbeat_front_addr":"172.21.15.181:6807/1683973278","state":["exists","up"]},{"osd":1,"uuid":"9726d641-8930-4b45-9192-6ee2b6bfcdcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6801","nonce":2136873375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6803","nonce":2136873375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6807","nonce":2136873375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6805","nonce":2136873375}]},"public_addr":"172.21.15.196:6801/2136873375","cluster_addr":"172.21.15.196:6803/2136873375","heartbeat_back_addr":"172.21.15.196:6807/2136873375","heartbeat_front_addr":"172.21.15.196:6805/2136873375","state":["exists","up"]},{"osd":2,"uuid":"3d5573cc-37c1-4c3b-a61e-2363ef3ff964","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6801","nonce":3291673137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6803","nonce":3291673137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6807","nonce":3291673137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6805","nonce":3291673137}]},"public_addr":"172.21.15.204:6801/3291673137","cluster_addr":"172.21.15.204:6803/3291673137","heartbeat_back_addr":"172.21.15.204:6807/3291673137","heartbeat_front_addr":"172.21.15.204:6805/3291673137","state":["exists","up"]},{"osd":3,"uuid":"b9b205e3-a335-4771-9a66-55cfe6a8c31a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6811","nonce":29064971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6813","nonce":29064971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6817","nonce":29064971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6815","nonce":29064971}]},"public_addr":"172.21.15.181:6811/29064971","cluster_addr":"172.21.15.181:6813/29064971","heartbeat_back_addr":"172.21.15.181:6817/29064971","heartbeat_front_addr":"172.21.15.181:6815/29064971","state":["exists","up"]},{"osd":4,"uuid":"550f01ef-1947-4f9f-b759-34e3ea831839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6809","nonce":1145408995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6811","nonce":1145408995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6815","nonce":1145408995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6813","nonce":1145408995}]},"public_addr":"172.21.15.196:6809/1145408995","cluster_addr":"172.21.15.196:6811/1145408995","heartbeat_back_addr":"172.21.15.196:6815/1145408995","heartbeat_front_addr":"172.21.15.196:6813/1145408995","state":["exists","up"]},{"osd":5,"uuid":"6b74cb3a-f39a-4131-8793-1d5286856aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6809","nonce":855680559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6811","nonce":855680559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6815","nonce":855680559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6813","nonce":855680559}]},"public_addr":"172.21.15.204:6809/855680559","cluster_addr":"172.21.15.204:6811/855680559","heartbeat_back_addr":"172.21.15.204:6815/855680559","heartbeat_front_addr":"172.21.15.204:6813/855680559","state":["exists","up"]},{"osd":6,"uuid":"dbe19bdb-08eb-40aa-842b-0710963d4497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6819","nonce":1561969338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6821","nonce":1561969338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6825","nonce":1561969338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6823","nonce":1561969338}]},"public_addr":"172.21.15.181:6819/1561969338","cluster_addr":"172.21.15.181:6821/1561969338","heartbeat_back_addr":"172.21.15.181:6825/1561969338","heartbeat_front_addr":"172.21.15.181:6823/1561969338","state":["exists","up"]},{"osd":7,"uuid":"016c145c-ad67-42ac-9775-ca2840790abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6817","nonce":824890163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6819","nonce":824890163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6823","nonce":824890163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6821","nonce":824890163}]},"public_addr":"172.21.15.196:6817/824890163","cluster_addr":"172.21.15.196:6819/824890163","heartbeat_back_addr":"172.21.15.196:6823/824890163","heartbeat_front_addr":"172.21.15.196:6821/824890163","state":["exists","up"]},{"osd":8,"uuid":"10235638-c1f1-41d3-8e24-f8b2344bfeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6817","nonce":3566909552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6819","nonce":3566909552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6823","nonce":3566909552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6821","nonce":3566909552}]},"public_addr":"172.21.15.204:6817/3566909552","cluster_addr":"172.21.15.204:6819/3566909552","heartbeat_back_addr":"172.21.15.204:6823/3566909552","heartbeat_front_addr":"172.21.15.204:6821/3566909552","state":["exists","up"]},{"osd":9,"uuid":"96ce5aa6-e6a9-4d83-8562-589ac94c8cd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6827","nonce":241693472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6829","nonce":241693472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6833","nonce":241693472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6831","nonce":241693472}]},"public_addr":"172.21.15.181:6827/241693472","cluster_addr":"172.21.15.181:6829/241693472","heartbeat_back_addr":"172.21.15.181:6833/241693472","heartbeat_front_addr":"172.21.15.181:6831/241693472","state":["exists","up"]},{"osd":10,"uuid":"d6c9ffab-731c-46d5-88cd-5f21ba5acfa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6825","nonce":3980645995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6827","nonce":3980645995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6831","nonce":3980645995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6829","nonce":3980645995}]},"public_addr":"172.21.15.196:6825/3980645995","cluster_addr":"172.21.15.196:6827/3980645995","heartbeat_back_addr":"172.21.15.196:6831/3980645995","heartbeat_front_addr":"172.21.15.196:6829/3980645995","state":["exists","up"]},{"osd":11,"uuid":"3f58a2d6-bbbb-4e4d-bf83-2c886dd9484b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6825","nonce":488710574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6827","nonce":488710574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6831","nonce":488710574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6829","nonce":488710574}]},"public_addr":"172.21.15.204:6825/488710574","cluster_addr":"172.21.15.204:6827/488710574","heartbeat_back_addr":"172.21.15.204:6831/488710574","heartbeat_front_addr":"172.21.15.204:6829/488710574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:28:09.700091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:28:33.838568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:28:57.265536+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:29:21.720066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:29:45.480447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:30:09.558384+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:30:34.253331+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:30:57.986111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:31:22.470746+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:31:47.548762+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:32:11.285657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:32:36.888004+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":6},{"from":7,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6800/2389231557":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/3398276284":"2024-05-04T20:26:06.561713+0000","172.21.15.181:6801/2389231557":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/2327733609":"2024-05-04T20:26:06.561713+0000","172.21.15.181:0/628540759":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/3387163329":"2024-05-04T20:26:06.561713+0000","172.21.15.181:6800/1802984663":"2024-05-04T20:26:06.561713+0000","172.21.15.181:0/67924109":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/3693389657":"2024-05-04T20:26:33.766772+0000","172.21.15.181:6801/1802984663":"2024-05-04T20:26:06.561713+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T20:35:46.532 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.534+0000 7f5c26ebe700 1 -- 172.21.15.181:0/3416612891 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5c100b2910 msgr2=0x7f5c100b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:46.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.534+0000 7f5c26ebe700 1 --2- 172.21.15.181:0/3416612891 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5c100b2910 0x7f5c100b4dd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c006fd0 tx=0x7f5c0c008040 comp rx=0 tx=0).stop 2024-05-03T20:35:46.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.534+0000 7f5c26ebe700 1 -- 172.21.15.181:0/3416612891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c201023a0 msgr2=0x7f5c2019beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:46.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.534+0000 7f5c26ebe700 1 --2- 172.21.15.181:0/3416612891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c201023a0 0x7f5c2019beb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5c140057f0 tx=0x7f5c14003680 comp rx=0 tx=0).stop 2024-05-03T20:35:46.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.534+0000 7f5c26ebe700 1 -- 172.21.15.181:0/3416612891 shutdown_connections 2024-05-03T20:35:46.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.534+0000 7f5c26ebe700 1 --2- 172.21.15.181:0/3416612891 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f5c100b2910 0x7f5c100b4dd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:46.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.534+0000 7f5c26ebe700 1 --2- 172.21.15.181:0/3416612891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c20108410 0x7f5c201a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:46.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.534+0000 7f5c26ebe700 1 --2- 172.21.15.181:0/3416612891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c201023a0 0x7f5c2019beb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:46.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.534+0000 7f5c26ebe700 1 --2- 172.21.15.181:0/3416612891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c20102d60 0x7f5c2019c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:46.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.534+0000 7f5c26ebe700 1 -- 172.21.15.181:0/3416612891 >> 172.21.15.181:0/3416612891 conn(0x7f5c20079350 msgr2=0x7f5c2010fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:46.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.534+0000 7f5c26ebe700 1 -- 172.21.15.181:0/3416612891 shutdown_connections 2024-05-03T20:35:46.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:46.534+0000 7f5c26ebe700 1 -- 172.21.15.181:0/3416612891 wait complete. 2024-05-03T20:35:47.222 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T20:35:47.462 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:35:47.822 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:47 smithi181 ceph-mon[29801]: pgmap v349: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-03T20:35:47.822 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:47 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3416612891' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T20:35:47.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:47 smithi196 ceph-mon[32858]: pgmap v349: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-03T20:35:47.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:47 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3416612891' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T20:35:47.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:47 smithi204 ceph-mon[33595]: pgmap v349: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-03T20:35:47.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:47 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3416612891' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T20:35:49.089 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.089+0000 7f54297e4700 1 -- 172.21.15.181:0/3853030519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5424108410 msgr2=0x7f54241087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:49.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.089+0000 7f54297e4700 1 --2- 172.21.15.181:0/3853030519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5424108410 0x7f54241087f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5414009990 tx=0x7f5414009ca0 comp rx=0 tx=0).stop 2024-05-03T20:35:49.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.089+0000 7f54297e4700 1 -- 172.21.15.181:0/3853030519 shutdown_connections 2024-05-03T20:35:49.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.089+0000 7f54297e4700 1 --2- 172.21.15.181:0/3853030519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5424102d60 0x7f542410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:49.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.089+0000 7f54297e4700 1 --2- 172.21.15.181:0/3853030519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5424108410 0x7f54241087f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:49.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.089+0000 7f54297e4700 1 --2- 172.21.15.181:0/3853030519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54241023a0 0x7f5424102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:49.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.089+0000 7f54297e4700 1 -- 172.21.15.181:0/3853030519 >> 172.21.15.181:0/3853030519 conn(0x7f5424079350 msgr2=0x7f5424079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:49.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.089+0000 7f54297e4700 1 -- 172.21.15.181:0/3853030519 shutdown_connections 2024-05-03T20:35:49.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.089+0000 7f54297e4700 1 -- 172.21.15.181:0/3853030519 wait complete. 2024-05-03T20:35:49.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.089+0000 7f54297e4700 1 Processor -- start 2024-05-03T20:35:49.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.090+0000 7f54297e4700 1 -- start start 2024-05-03T20:35:49.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.090+0000 7f54297e4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54241023a0 0x7f542419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:49.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.090+0000 7f54297e4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5424102d60 0x7f542419afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:49.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.090+0000 7f54297e4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5424108410 0x7f54241a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:49.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.090+0000 7f54297e4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f54241125e0 con 0x7f5424102d60 2024-05-03T20:35:49.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.090+0000 7f54297e4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5424112460 con 0x7f54241023a0 2024-05-03T20:35:49.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.090+0000 7f54297e4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5424112760 con 0x7f5424108410 2024-05-03T20:35:49.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.090+0000 7f541a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5424102d60 0x7f542419afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:49.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.090+0000 7f541a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5424102d60 0x7f542419afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42644/0 (socket says 172.21.15.181:42644) 2024-05-03T20:35:49.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.090+0000 7f541a7fc700 1 -- 172.21.15.181:0/3632358045 learned_addr learned my addr 172.21.15.181:0/3632358045 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:49.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.090+0000 7f54237fe700 1 --2- 172.21.15.181:0/3632358045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5424108410 0x7f54241a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:49.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.090+0000 7f5422ffd700 1 --2- 172.21.15.181:0/3632358045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54241023a0 0x7f542419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:49.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.090+0000 7f541a7fc700 1 -- 172.21.15.181:0/3632358045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5424108410 msgr2=0x7f54241a5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:49.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.090+0000 7f541a7fc700 1 --2- 172.21.15.181:0/3632358045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5424108410 0x7f54241a5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:49.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.090+0000 7f541a7fc700 1 -- 172.21.15.181:0/3632358045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54241023a0 msgr2=0x7f542419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:49.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.090+0000 7f541a7fc700 1 --2- 172.21.15.181:0/3632358045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54241023a0 0x7f542419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:49.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.090+0000 7f541a7fc700 1 -- 172.21.15.181:0/3632358045 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54241a6020 con 0x7f5424102d60 2024-05-03T20:35:49.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.090+0000 7f5422ffd700 1 --2- 172.21.15.181:0/3632358045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54241023a0 0x7f542419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:35:49.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.090+0000 7f541a7fc700 1 --2- 172.21.15.181:0/3632358045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5424102d60 0x7f542419afa0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f540c00c970 tx=0x7f540c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:49.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.091+0000 7f5420ff9700 1 -- 172.21.15.181:0/3632358045 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c007aa0 con 0x7f5424102d60 2024-05-03T20:35:49.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.091+0000 7f5420ff9700 1 -- 172.21.15.181:0/3632358045 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f540c004ce0 con 0x7f5424102d60 2024-05-03T20:35:49.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.091+0000 7f54297e4700 1 -- 172.21.15.181:0/3632358045 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54241a6250 con 0x7f5424102d60 2024-05-03T20:35:49.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.091+0000 7f54297e4700 1 -- 172.21.15.181:0/3632358045 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54241a6710 con 0x7f5424102d60 2024-05-03T20:35:49.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.091+0000 7f5420ff9700 1 -- 172.21.15.181:0/3632358045 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c005510 con 0x7f5424102d60 2024-05-03T20:35:49.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.092+0000 7f54297e4700 1 -- 172.21.15.181:0/3632358045 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f542406a190 con 0x7f5424102d60 2024-05-03T20:35:49.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.094+0000 7f5420ff9700 1 -- 172.21.15.181:0/3632358045 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f540c007c40 con 0x7f5424102d60 2024-05-03T20:35:49.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.094+0000 7f5420ff9700 1 --2- 172.21.15.181:0/3632358045 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f54100b2910 0x7f54100b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:49.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.094+0000 7f5422ffd700 1 --2- 172.21.15.181:0/3632358045 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f54100b2910 0x7f54100b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:49.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.094+0000 7f5420ff9700 1 -- 172.21.15.181:0/3632358045 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f540c0e2370 con 0x7f5424102d60 2024-05-03T20:35:49.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.095+0000 7f5422ffd700 1 --2- 172.21.15.181:0/3632358045 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f54100b2910 0x7f54100b4dd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5414006010 tx=0x7f541400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:49.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.098+0000 7f5420ff9700 1 -- 172.21.15.181:0/3632358045 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f540c0aa050 con 0x7f5424102d60 2024-05-03T20:35:49.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.353+0000 7f54297e4700 1 -- 172.21.15.181:0/3632358045 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f542402e510 con 0x7f5424102d60 2024-05-03T20:35:49.352 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.353+0000 7f5420ff9700 1 -- 172.21.15.181:0/3632358045 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11954 (secure 0 0 0) 0x7f540c0a9950 con 0x7f5424102d60 2024-05-03T20:35:49.352 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:35:49.352 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2414172454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2414172454},{"type":"v1","addr":"172.21.15.181:6835","nonce":2414172454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/204127189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":204127189},{"type":"v1","addr":"172.21.15.181:6837","nonce":204127189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T20:33:42.487965+0000","modified":"2024-05-03T20:34:00.359735+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24475,"mds_2":24464,"mds_3":24463,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14562":{"gid":14562,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6839/257361276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":257361276},{"type":"v1","addr":"172.21.15.181:6839","nonce":257361276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6841/2403160115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2403160115},{"type":"v1","addr":"172.21.15.181:6841","nonce":2403160115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3975688022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3975688022},{"type":"v1","addr":"172.21.15.196:6833","nonce":3975688022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2409628823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2409628823},{"type":"v1","addr":"172.21.15.204:6833","nonce":2409628823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1820206106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1820206106},{"type":"v1","addr":"172.21.15.196:6835","nonce":1820206106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6835/3870850287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3870850287},{"type":"v1","addr":"172.21.15.204:6835","nonce":3870850287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/3006728559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3006728559},{"type":"v1","addr":"172.21.15.204:6837","nonce":3006728559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6837/1529420259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1529420259},{"type":"v1","addr":"172.21.15.196:6837","nonce":1529420259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/1644828559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1644828559},{"type":"v1","addr":"172.21.15.204:6839","nonce":1644828559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.196:6839/1182826743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1182826743},{"type":"v1","addr":"172.21.15.196:6839","nonce":1182826743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24457,24464,24463,24475,24488]},"id":1}]} 2024-05-03T20:35:49.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.356+0000 7f54297e4700 1 -- 172.21.15.181:0/3632358045 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f54100b2910 msgr2=0x7f54100b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:49.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.356+0000 7f54297e4700 1 --2- 172.21.15.181:0/3632358045 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f54100b2910 0x7f54100b4dd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5414006010 tx=0x7f541400b540 comp rx=0 tx=0).stop 2024-05-03T20:35:49.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.356+0000 7f54297e4700 1 -- 172.21.15.181:0/3632358045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5424102d60 msgr2=0x7f542419afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:49.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.356+0000 7f54297e4700 1 --2- 172.21.15.181:0/3632358045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5424102d60 0x7f542419afa0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f540c00c970 tx=0x7f540c00cc80 comp rx=0 tx=0).stop 2024-05-03T20:35:49.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.356+0000 7f54297e4700 1 -- 172.21.15.181:0/3632358045 shutdown_connections 2024-05-03T20:35:49.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.356+0000 7f54297e4700 1 --2- 172.21.15.181:0/3632358045 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f54100b2910 0x7f54100b4dd0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:49.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.356+0000 7f54297e4700 1 --2- 172.21.15.181:0/3632358045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5424108410 0x7f54241a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:49.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.356+0000 7f54297e4700 1 --2- 172.21.15.181:0/3632358045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54241023a0 0x7f542419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:49.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.356+0000 7f54297e4700 1 --2- 172.21.15.181:0/3632358045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5424102d60 0x7f542419afa0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:49.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.356+0000 7f54297e4700 1 -- 172.21.15.181:0/3632358045 >> 172.21.15.181:0/3632358045 conn(0x7f5424079350 msgr2=0x7f5424101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:49.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.356+0000 7f54297e4700 1 -- 172.21.15.181:0/3632358045 shutdown_connections 2024-05-03T20:35:49.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:49.356+0000 7f54297e4700 1 -- 172.21.15.181:0/3632358045 wait complete. 2024-05-03T20:35:49.355 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 27 2024-05-03T20:35:49.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:49 smithi181 ceph-mon[29801]: pgmap v350: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:35:49.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:49 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3632358045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:35:49.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:49 smithi196 ceph-mon[32858]: pgmap v350: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:35:49.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:49 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3632358045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:35:49.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:49 smithi204 ceph-mon[33595]: pgmap v350: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:35:49.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:49 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3632358045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:35:50.045 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-03T20:33:42.487965+0000', 'modified': '2024-05-03T20:34:00.359735+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24488, 'mds_1': 24475, 'mds_2': 24464, 'mds_3': 24463, 'mds_4': 24457}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14562': {'gid': 14562, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6839/257361276', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 257361276}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 257361276}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14574': {'gid': 14574, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6841/2403160115', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6840', 'nonce': 2403160115}, {'type': 'v1', 'addr': '172.21.15.181:6841', 'nonce': 2403160115}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24410': {'gid': 24410, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6833/3975688022', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 3975688022}, {'type': 'v1', 'addr': '172.21.15.196:6833', 'nonce': 3975688022}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6833/2409628823', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6832', 'nonce': 2409628823}, {'type': 'v1', 'addr': '172.21.15.204:6833', 'nonce': 2409628823}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24437': {'gid': 24437, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6835/1820206106', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 1820206106}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 1820206106}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.204:6835/3870850287', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 3870850287}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 3870850287}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.204:6837/3006728559', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 3006728559}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 3006728559}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24464': {'gid': 24464, 'name': 'h', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.196:6837/1529420259', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6836', 'nonce': 1529420259}, {'type': 'v1', 'addr': '172.21.15.196:6837', 'nonce': 1529420259}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.204:6839/1644828559', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 1644828559}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 1644828559}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.196:6839/1182826743', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6838', 'nonce': 1182826743}, {'type': 'v1', 'addr': '172.21.15.196:6839', 'nonce': 1182826743}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24488, 'qdb_cluster': [24457, 24464, 24463, 24475, 24488]} 2024-05-03T20:35:50.046 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-03T20:35:50.046 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph tell mds.f status 2024-05-03T20:35:50.285 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:35:51.802 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:51 smithi181 ceph-mon[29801]: pgmap v351: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:35:51.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:51 smithi196 ceph-mon[32858]: pgmap v351: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:35:51.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:51 smithi204 ceph-mon[33595]: pgmap v351: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:35:52.016 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.017+0000 7fd43548a700 1 -- 172.21.15.181:0/3343211606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd430103750 msgr2=0x7fd430103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:52.016 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.017+0000 7fd43548a700 1 --2- 172.21.15.181:0/3343211606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd430103750 0x7fd430103b30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd4200099c0 tx=0x7fd420009cd0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.016 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.018+0000 7fd43548a700 1 -- 172.21.15.181:0/3343211606 shutdown_connections 2024-05-03T20:35:52.016 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.018+0000 7fd43548a700 1 --2- 172.21.15.181:0/3343211606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd430108850 0x7fd4301103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.016 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.018+0000 7fd43548a700 1 --2- 172.21.15.181:0/3343211606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd430103750 0x7fd430103b30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.016 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.018+0000 7fd43548a700 1 --2- 172.21.15.181:0/3343211606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd430104100 0x7fd430108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.017 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.018+0000 7fd43548a700 1 -- 172.21.15.181:0/3343211606 >> 172.21.15.181:0/3343211606 conn(0x7fd43007a600 msgr2=0x7fd43007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:52.017 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.018+0000 7fd43548a700 1 -- 172.21.15.181:0/3343211606 shutdown_connections 2024-05-03T20:35:52.017 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.018+0000 7fd43548a700 1 -- 172.21.15.181:0/3343211606 wait complete. 2024-05-03T20:35:52.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.018+0000 7fd43548a700 1 Processor -- start 2024-05-03T20:35:52.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.018+0000 7fd43548a700 1 -- start start 2024-05-03T20:35:52.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.019+0000 7fd43548a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd430103750 0x7fd43019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:52.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.019+0000 7fd43548a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd430104100 0x7fd43019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:52.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.019+0000 7fd43548a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd430108850 0x7fd430199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:52.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.019+0000 7fd43548a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd430113640 con 0x7fd430103750 2024-05-03T20:35:52.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.019+0000 7fd43548a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd4301134c0 con 0x7fd430104100 2024-05-03T20:35:52.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.019+0000 7fd43548a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd4301137c0 con 0x7fd430108850 2024-05-03T20:35:52.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.019+0000 7fd42effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd430103750 0x7fd43019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:52.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.019+0000 7fd42effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd430103750 0x7fd43019f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42662/0 (socket says 172.21.15.181:42662) 2024-05-03T20:35:52.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.019+0000 7fd42effd700 1 -- 172.21.15.181:0/3067824757 learned_addr learned my addr 172.21.15.181:0/3067824757 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:52.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.019+0000 7fd42e7fc700 1 --2- 172.21.15.181:0/3067824757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd430104100 0x7fd43019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:52.020 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.019+0000 7fd42f7fe700 1 --2- 172.21.15.181:0/3067824757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd430108850 0x7fd430199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:52.020 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.019+0000 7fd42effd700 1 -- 172.21.15.181:0/3067824757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd430108850 msgr2=0x7fd430199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:52.020 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.019+0000 7fd42effd700 1 --2- 172.21.15.181:0/3067824757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd430108850 0x7fd430199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.020 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.019+0000 7fd42effd700 1 -- 172.21.15.181:0/3067824757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd430104100 msgr2=0x7fd43019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:52.020 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.019+0000 7fd42effd700 1 --2- 172.21.15.181:0/3067824757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd430104100 0x7fd43019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.020 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.019+0000 7fd42effd700 1 -- 172.21.15.181:0/3067824757 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43019a080 con 0x7fd430103750 2024-05-03T20:35:52.020 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.019+0000 7fd42effd700 1 --2- 172.21.15.181:0/3067824757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd430103750 0x7fd43019f760 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd42000b580 tx=0x7fd4200047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:52.020 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.019+0000 7fd417fff700 1 -- 172.21.15.181:0/3067824757 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd420027070 con 0x7fd430103750 2024-05-03T20:35:52.020 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.019+0000 7fd43548a700 1 -- 172.21.15.181:0/3067824757 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd43019a310 con 0x7fd430103750 2024-05-03T20:35:52.020 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.019+0000 7fd417fff700 1 -- 172.21.15.181:0/3067824757 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd420003f20 con 0x7fd430103750 2024-05-03T20:35:52.020 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.019+0000 7fd417fff700 1 -- 172.21.15.181:0/3067824757 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd420022d70 con 0x7fd430103750 2024-05-03T20:35:52.020 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.019+0000 7fd43548a700 1 -- 172.21.15.181:0/3067824757 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4301a6630 con 0x7fd430103750 2024-05-03T20:35:52.022 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.022+0000 7fd417fff700 1 -- 172.21.15.181:0/3067824757 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fd420022500 con 0x7fd430103750 2024-05-03T20:35:52.022 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.022+0000 7fd417fff700 1 --2- 172.21.15.181:0/3067824757 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd4180b2910 0x7fd4180b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:52.022 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.022+0000 7fd42e7fc700 1 --2- 172.21.15.181:0/3067824757 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd4180b2910 0x7fd4180b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:52.022 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.023+0000 7fd417fff700 1 -- 172.21.15.181:0/3067824757 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7fd4200e2fa0 con 0x7fd430103750 2024-05-03T20:35:52.022 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.023+0000 7fd42e7fc700 1 --2- 172.21.15.181:0/3067824757 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd4180b2910 0x7fd4180b4dd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd4301030a0 tx=0x7fd42400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:52.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.023+0000 7fd43fd1db80 1 Processor -- start 2024-05-03T20:35:52.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.023+0000 7fd43fd1db80 1 -- start start 2024-05-03T20:35:52.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.023+0000 7fd43fd1db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d92f4c10 0x5576d92f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:52.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.023+0000 7fd43fd1db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576d92f5530 0x5576d92f8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:52.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.023+0000 7fd43fd1db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f9230 0x5576d92f9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:52.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.023+0000 7fd43fd1db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5576d9247810 con 0x5576d92f9230 2024-05-03T20:35:52.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.023+0000 7fd43fd1db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5576d91e6770 con 0x5576d92f5530 2024-05-03T20:35:52.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.023+0000 7fd43fd1db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5576d92437d0 con 0x5576d92f4c10 2024-05-03T20:35:52.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.023+0000 7fd42f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f9230 0x5576d92f9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:52.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.023+0000 7fd42f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f9230 0x5576d92f9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42670/0 (socket says 172.21.15.181:42670) 2024-05-03T20:35:52.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.023+0000 7fd42f7fe700 1 -- 172.21.15.181:0/2002153400 learned_addr learned my addr 172.21.15.181:0/2002153400 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:52.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.023+0000 7fd42e7fc700 1 --2- 172.21.15.181:0/2002153400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576d92f5530 0x5576d92f8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:52.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.023+0000 7fd42effd700 1 --2- 172.21.15.181:0/2002153400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d92f4c10 0x5576d92f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:52.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.024+0000 7fd42f7fe700 1 -- 172.21.15.181:0/2002153400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d92f4c10 msgr2=0x5576d92f4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:35:52.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.024+0000 7fd42f7fe700 1 --2- 172.21.15.181:0/2002153400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d92f4c10 0x5576d92f4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.024+0000 7fd42f7fe700 1 -- 172.21.15.181:0/2002153400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576d92f5530 msgr2=0x5576d92f8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:35:52.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.024+0000 7fd42f7fe700 1 --2- 172.21.15.181:0/2002153400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576d92f5530 0x5576d92f8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.024+0000 7fd42f7fe700 1 -- 172.21.15.181:0/2002153400 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576d9194880 con 0x5576d92f9230 2024-05-03T20:35:52.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.024+0000 7fd42f7fe700 1 --2- 172.21.15.181:0/2002153400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f9230 0x5576d92f9690 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd428007430 tx=0x7fd428007740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce31c5951281c010 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:52.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.024+0000 7fd4157fa700 1 -- 172.21.15.181:0/2002153400 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd428018070 con 0x5576d92f9230 2024-05-03T20:35:52.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.024+0000 7fd4157fa700 1 -- 172.21.15.181:0/2002153400 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd428013ab0 con 0x5576d92f9230 2024-05-03T20:35:52.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.024+0000 7fd4157fa700 1 -- 172.21.15.181:0/2002153400 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd428012ac0 con 0x5576d92f9230 2024-05-03T20:35:52.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.025+0000 7fd43fd1db80 1 -- 172.21.15.181:0/2002153400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f9230 msgr2=0x5576d92f9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:52.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.025+0000 7fd43fd1db80 1 --2- 172.21.15.181:0/2002153400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f9230 0x5576d92f9690 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd428007430 tx=0x7fd428007740 comp rx=0 tx=0).stop 2024-05-03T20:35:52.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.025+0000 7fd43fd1db80 1 -- 172.21.15.181:0/2002153400 shutdown_connections 2024-05-03T20:35:52.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.025+0000 7fd43fd1db80 1 --2- 172.21.15.181:0/2002153400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f9230 0x5576d92f9690 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.025+0000 7fd43fd1db80 1 --2- 172.21.15.181:0/2002153400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d92f4c10 0x5576d92f4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.025+0000 7fd43fd1db80 1 --2- 172.21.15.181:0/2002153400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576d92f5530 0x5576d92f8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.025+0000 7fd43fd1db80 1 -- 172.21.15.181:0/2002153400 >> 172.21.15.181:0/2002153400 conn(0x5576d92210b0 msgr2=0x5576d92ef280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:52.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.025+0000 7fd43fd1db80 1 -- 172.21.15.181:0/2002153400 shutdown_connections 2024-05-03T20:35:52.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.025+0000 7fd43fd1db80 1 -- 172.21.15.181:0/2002153400 wait complete. 2024-05-03T20:35:52.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.025+0000 7fd43fd1db80 1 -- start start 2024-05-03T20:35:52.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.025+0000 7fd43fd1db80 1 Processor -- start 2024-05-03T20:35:52.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.025+0000 7fd43fd1db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d92f4c10 0x5576d9300660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:52.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.025+0000 7fd43fd1db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576d92f5530 0x5576d92fda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:52.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.025+0000 7fd43fd1db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f9230 0x5576d92fdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:52.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.025+0000 7fd43fd1db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5576d9301490 con 0x5576d92f9230 2024-05-03T20:35:52.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.025+0000 7fd43fd1db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5576d91adbe0 con 0x5576d92f5530 2024-05-03T20:35:52.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.025+0000 7fd43fd1db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5576d9301610 con 0x5576d92f4c10 2024-05-03T20:35:52.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.025+0000 7fd43fd1db80 10 client.0 _ll_register_callbacks cb 0x5576d91df8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T20:35:52.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.026+0000 7fd42f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f9230 0x5576d92fdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:52.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.026+0000 7fd42f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f9230 0x5576d92fdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42680/0 (socket says 172.21.15.181:42680) 2024-05-03T20:35:52.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.026+0000 7fd42f7fe700 1 -- 172.21.15.181:0/1281571818 learned_addr learned my addr 172.21.15.181:0/1281571818 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:52.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.026+0000 7fd42effd700 1 --2- 172.21.15.181:0/1281571818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d92f4c10 0x5576d9300660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:52.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.026+0000 7fd42f7fe700 1 -- 172.21.15.181:0/1281571818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d92f4c10 msgr2=0x5576d9300660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:35:52.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.026+0000 7fd42f7fe700 1 --2- 172.21.15.181:0/1281571818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d92f4c10 0x5576d9300660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.026+0000 7fd42e7fc700 1 --2- 172.21.15.181:0/1281571818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576d92f5530 0x5576d92fda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:52.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.026+0000 7fd42f7fe700 1 -- 172.21.15.181:0/1281571818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576d92f5530 msgr2=0x5576d92fda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:35:52.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.026+0000 7fd42f7fe700 1 --2- 172.21.15.181:0/1281571818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576d92f5530 0x5576d92fda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.026+0000 7fd42f7fe700 1 -- 172.21.15.181:0/1281571818 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576d91e3640 con 0x5576d92f9230 2024-05-03T20:35:52.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.026+0000 7fd42f7fe700 1 --2- 172.21.15.181:0/1281571818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f9230 0x5576d92fdf40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd428012640 tx=0x7fd428008540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe972472ef91556 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:52.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.026+0000 7fd412ff5700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-03T20:35:52.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.026+0000 7fd412ff5700 1 -- 172.21.15.181:0/1281571818 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd428018040 con 0x5576d92f9230 2024-05-03T20:35:52.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.026+0000 7fd412ff5700 1 -- 172.21.15.181:0/1281571818 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7fd428013840 con 0x5576d92f9230 2024-05-03T20:35:52.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.026+0000 7fd43fd1db80 1 -- 172.21.15.181:0/1281571818 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd428013ab0 con 0x5576d92f9230 2024-05-03T20:35:52.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.026+0000 7fd412ff5700 1 -- 172.21.15.181:0/1281571818 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd428004a90 con 0x5576d92f9230 2024-05-03T20:35:52.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.026+0000 7fd43fd1db80 10 client.14928 fetch_fsmap learned FSMap version 27 2024-05-03T20:35:52.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.026+0000 7fd43fd1db80 1 -- 172.21.15.181:0/1281571818 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5576d930bc90 con 0x5576d92f9230 2024-05-03T20:35:52.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.027+0000 7fd412ff5700 1 -- 172.21.15.181:0/1281571818 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd428013ab0 con 0x5576d92f9230 2024-05-03T20:35:52.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.027+0000 7fd412ff5700 1 -- 172.21.15.181:0/1281571818 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd4280050e0 con 0x5576d92f9230 2024-05-03T20:35:52.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.027+0000 7fd43fd1db80 10 client.14928 fetch_fsmap finished waiting for FSMap version 27 2024-05-03T20:35:52.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.027+0000 7fd43fd1db80 10 client.14928 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-03T20:35:52.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.027+0000 7fd43fd1db80 20 client.14928 populate_metadata read hostname 'smithi181' 2024-05-03T20:35:52.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.027+0000 7fd43fd1db80 1 --2- 172.21.15.181:0/1281571818 >> [v2:172.21.15.204:6834/3870850287,v1:172.21.15.204:6835/3870850287] conn(0x5576d930e0a0 0x5576d930f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:52.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.027+0000 7fd43fd1db80 4 client.14928 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T20:35:52.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.027+0000 7fd43fd1db80 1 -- 172.21.15.181:0/1281571818 --> [v2:172.21.15.204:6834/3870850287,v1:172.21.15.204:6835/3870850287] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576d92551f0 con 0x5576d930e0a0 2024-05-03T20:35:52.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.027+0000 7fd42effd700 1 --2- 172.21.15.181:0/1281571818 >> [v2:172.21.15.204:6834/3870850287,v1:172.21.15.204:6835/3870850287] conn(0x5576d930e0a0 0x5576d930f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:52.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.027+0000 7fd412ff5700 10 client.14928 ms_handle_connect on v2:172.21.15.204:6834/3870850287 2024-05-03T20:35:52.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.027+0000 7fd42effd700 1 --2- 172.21.15.181:0/1281571818 >> [v2:172.21.15.204:6834/3870850287,v1:172.21.15.204:6835/3870850287] conn(0x5576d930e0a0 0x5576d930f510 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f1eb8cc17722021 server_cookie=ce9273b6278ef8fd in_seq=0 out_seq=0 2024-05-03T20:35:52.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.028+0000 7fd412ff5700 1 -- 172.21.15.181:0/1281571818 <== mds.4 v2:172.21.15.204:6834/3870850287 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5576d92551f0 con 0x5576d930e0a0 2024-05-03T20:35:52.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.028+0000 7fd412ff5700 10 client.14928 handle_command_reply: tid=0 2024-05-03T20:35:52.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.028+0000 7fd43fd1db80 1 client.14928 shutdown 2024-05-03T20:35:52.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.028+0000 7fd43fd1db80 1 -- 172.21.15.181:0/1281571818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f9230 msgr2=0x5576d92fdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:52.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.028+0000 7fd43fd1db80 1 --2- 172.21.15.181:0/1281571818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f9230 0x5576d92fdf40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd428012640 tx=0x7fd428008540 comp rx=0 tx=0).stop 2024-05-03T20:35:52.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.028+0000 7fd43fd1db80 1 -- 172.21.15.181:0/1281571818 shutdown_connections 2024-05-03T20:35:52.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.028+0000 7fd43fd1db80 1 --2- 172.21.15.181:0/1281571818 >> [v2:172.21.15.204:6834/3870850287,v1:172.21.15.204:6835/3870850287] conn(0x5576d930e0a0 0x5576d930f510 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.028+0000 7fd43fd1db80 1 --2- 172.21.15.181:0/1281571818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f9230 0x5576d92fdf40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.028+0000 7fd43fd1db80 1 --2- 172.21.15.181:0/1281571818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d92f4c10 0x5576d9300660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.028+0000 7fd412ff5700 0 client.14928 ms_handle_reset on v2:172.21.15.204:6834/3870850287 2024-05-03T20:35:52.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.028+0000 7fd43fd1db80 1 --2- 172.21.15.181:0/1281571818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576d92f5530 0x5576d92fda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.028+0000 7fd43fd1db80 1 -- 172.21.15.181:0/1281571818 >> 172.21.15.181:0/1281571818 conn(0x5576d92210b0 msgr2=0x5576d92ee8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:52.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.028+0000 7fd43fd1db80 1 -- 172.21.15.181:0/1281571818 shutdown_connections 2024-05-03T20:35:52.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.029+0000 7fd43fd1db80 1 -- 172.21.15.181:0/1281571818 wait complete. 2024-05-03T20:35:52.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.029+0000 7fd43fd1db80 20 client.14928 trim_cache size 0 max 16384 2024-05-03T20:35:52.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.049+0000 7fd43fd1db80 1 Processor -- start 2024-05-03T20:35:52.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.049+0000 7fd43fd1db80 1 -- start start 2024-05-03T20:35:52.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.049+0000 7fd43fd1db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d92f4c10 0x5576d92ef070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:52.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.049+0000 7fd43fd1db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f5530 0x5576d92f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:52.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.049+0000 7fd43fd1db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576d92f9230 0x5576d92f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:52.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.049+0000 7fd43fd1db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5576d930d850 con 0x5576d92f5530 2024-05-03T20:35:52.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.049+0000 7fd43fd1db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5576d930df20 con 0x5576d92f9230 2024-05-03T20:35:52.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.049+0000 7fd43fd1db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5576d930e5f0 con 0x5576d92f4c10 2024-05-03T20:35:52.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.049+0000 7fd42e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f5530 0x5576d92f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:52.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.049+0000 7fd42e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f5530 0x5576d92f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42690/0 (socket says 172.21.15.181:42690) 2024-05-03T20:35:52.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.049+0000 7fd42e7fc700 1 -- 172.21.15.181:0/2052171570 learned_addr learned my addr 172.21.15.181:0/2052171570 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:52.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.050+0000 7fd42e7fc700 1 -- 172.21.15.181:0/2052171570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d92f4c10 msgr2=0x5576d92ef070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:35:52.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.050+0000 7fd42e7fc700 1 --2- 172.21.15.181:0/2052171570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d92f4c10 0x5576d92ef070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.050+0000 7fd42f7fe700 1 --2- 172.21.15.181:0/2052171570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576d92f9230 0x5576d92f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:52.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.050+0000 7fd42e7fc700 1 -- 172.21.15.181:0/2052171570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576d92f9230 msgr2=0x5576d92f71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:35:52.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.050+0000 7fd42e7fc700 1 --2- 172.21.15.181:0/2052171570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576d92f9230 0x5576d92f71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.050+0000 7fd42e7fc700 1 -- 172.21.15.181:0/2052171570 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576d92efd10 con 0x5576d92f5530 2024-05-03T20:35:52.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.050+0000 7fd42f7fe700 1 --2- 172.21.15.181:0/2052171570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576d92f9230 0x5576d92f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:35:52.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.050+0000 7fd42e7fc700 1 --2- 172.21.15.181:0/2052171570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f5530 0x5576d92f6c60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd424010f80 tx=0x7fd424013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad3967e472f08e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:52.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.050+0000 7fd414ff9700 1 -- 172.21.15.181:0/2052171570 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424028630 con 0x5576d92f5530 2024-05-03T20:35:52.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.050+0000 7fd414ff9700 1 -- 172.21.15.181:0/2052171570 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd42401e8d0 con 0x5576d92f5530 2024-05-03T20:35:52.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.050+0000 7fd414ff9700 1 -- 172.21.15.181:0/2052171570 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424027840 con 0x5576d92f5530 2024-05-03T20:35:52.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.051+0000 7fd43fd1db80 1 -- 172.21.15.181:0/2052171570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f5530 msgr2=0x5576d92f6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:52.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.051+0000 7fd43fd1db80 1 --2- 172.21.15.181:0/2052171570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f5530 0x5576d92f6c60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd424010f80 tx=0x7fd424013000 comp rx=0 tx=0).stop 2024-05-03T20:35:52.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.051+0000 7fd43fd1db80 1 -- 172.21.15.181:0/2052171570 shutdown_connections 2024-05-03T20:35:52.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.051+0000 7fd43fd1db80 1 --2- 172.21.15.181:0/2052171570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576d92f9230 0x5576d92f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.051+0000 7fd43fd1db80 1 --2- 172.21.15.181:0/2052171570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d92f4c10 0x5576d92ef070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.051+0000 7fd43fd1db80 1 --2- 172.21.15.181:0/2052171570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f5530 0x5576d92f6c60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.051+0000 7fd43fd1db80 1 -- 172.21.15.181:0/2052171570 >> 172.21.15.181:0/2052171570 conn(0x5576d92210b0 msgr2=0x5576d92fc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:52.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.051+0000 7fd43fd1db80 1 -- 172.21.15.181:0/2052171570 shutdown_connections 2024-05-03T20:35:52.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.051+0000 7fd43fd1db80 1 -- 172.21.15.181:0/2052171570 wait complete. 2024-05-03T20:35:52.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.051+0000 7fd43fd1db80 1 -- start start 2024-05-03T20:35:52.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.051+0000 7fd43fd1db80 1 Processor -- start 2024-05-03T20:35:52.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.051+0000 7fd43fd1db80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576d92f4c10 0x5576d92fb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:52.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.051+0000 7fd43fd1db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f5530 0x5576d9318980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:52.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.051+0000 7fd43fd1db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d92f9230 0x5576d9318ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:52.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.051+0000 7fd43fd1db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5576d930f390 con 0x5576d92f5530 2024-05-03T20:35:52.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.051+0000 7fd43fd1db80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5576d930ecc0 con 0x5576d92f4c10 2024-05-03T20:35:52.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.051+0000 7fd43fd1db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5576d930fa60 con 0x5576d92f9230 2024-05-03T20:35:52.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.051+0000 7fd43fd1db80 10 client.0 _ll_register_callbacks cb 0x5576d91df8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T20:35:52.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.051+0000 7fd42e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f5530 0x5576d9318980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:52.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.051+0000 7fd42e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f5530 0x5576d9318980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42702/0 (socket says 172.21.15.181:42702) 2024-05-03T20:35:52.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.051+0000 7fd42e7fc700 1 -- 172.21.15.181:0/114214962 learned_addr learned my addr 172.21.15.181:0/114214962 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:52.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.052+0000 7fd42f7fe700 1 --2- 172.21.15.181:0/114214962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d92f9230 0x5576d9318ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:52.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.052+0000 7fd42effd700 1 --2- 172.21.15.181:0/114214962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576d92f4c10 0x5576d92fb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:52.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.052+0000 7fd42e7fc700 1 -- 172.21.15.181:0/114214962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d92f9230 msgr2=0x5576d9318ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:35:52.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.052+0000 7fd42e7fc700 1 --2- 172.21.15.181:0/114214962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d92f9230 0x5576d9318ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.052+0000 7fd42e7fc700 1 -- 172.21.15.181:0/114214962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576d92f4c10 msgr2=0x5576d92fb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:35:52.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.052+0000 7fd42e7fc700 1 --2- 172.21.15.181:0/114214962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576d92f4c10 0x5576d92fb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.052+0000 7fd42e7fc700 1 -- 172.21.15.181:0/114214962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576d92f63f0 con 0x5576d92f5530 2024-05-03T20:35:52.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.052+0000 7fd42f7fe700 1 --2- 172.21.15.181:0/114214962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d92f9230 0x5576d9318ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:35:52.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.052+0000 7fd42e7fc700 1 --2- 172.21.15.181:0/114214962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f5530 0x5576d9318980 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd424005d90 tx=0x7fd42401e7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cae23c486a61f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:52.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.052+0000 7fd412ff5700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-03T20:35:52.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.052+0000 7fd412ff5700 1 -- 172.21.15.181:0/114214962 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424037060 con 0x5576d92f5530 2024-05-03T20:35:52.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.052+0000 7fd412ff5700 1 -- 172.21.15.181:0/114214962 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7fd424005040 con 0x5576d92f5530 2024-05-03T20:35:52.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.052+0000 7fd43fd1db80 1 -- 172.21.15.181:0/114214962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd42401e8d0 con 0x5576d92f5530 2024-05-03T20:35:52.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.052+0000 7fd412ff5700 1 -- 172.21.15.181:0/114214962 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd424005240 con 0x5576d92f5530 2024-05-03T20:35:52.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.052+0000 7fd43fd1db80 10 client.14934 fetch_fsmap learned FSMap version 27 2024-05-03T20:35:52.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.052+0000 7fd43fd1db80 1 -- 172.21.15.181:0/114214962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5576d9379860 con 0x5576d92f5530 2024-05-03T20:35:52.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.052+0000 7fd412ff5700 1 -- 172.21.15.181:0/114214962 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd42401e8d0 con 0x5576d92f5530 2024-05-03T20:35:52.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.052+0000 7fd412ff5700 1 -- 172.21.15.181:0/114214962 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd42403c070 con 0x5576d92f5530 2024-05-03T20:35:52.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.053+0000 7fd43fd1db80 10 client.14934 fetch_fsmap finished waiting for FSMap version 27 2024-05-03T20:35:52.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.053+0000 7fd43fd1db80 10 client.14934 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-03T20:35:52.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.053+0000 7fd43fd1db80 20 client.14934 populate_metadata read hostname 'smithi181' 2024-05-03T20:35:52.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.053+0000 7fd43fd1db80 1 --2- 172.21.15.181:0/114214962 >> [v2:172.21.15.204:6834/3870850287,v1:172.21.15.204:6835/3870850287] conn(0x5576d937e2a0 0x5576d937cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:52.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.053+0000 7fd43fd1db80 4 client.14934 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T20:35:52.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.053+0000 7fd43fd1db80 1 -- 172.21.15.181:0/114214962 --> [v2:172.21.15.204:6834/3870850287,v1:172.21.15.204:6835/3870850287] -- command(tid 0: {"prefix": "status"}) -- 0x5576d924f290 con 0x5576d937e2a0 2024-05-03T20:35:52.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.053+0000 7fd42effd700 1 --2- 172.21.15.181:0/114214962 >> [v2:172.21.15.204:6834/3870850287,v1:172.21.15.204:6835/3870850287] conn(0x5576d937e2a0 0x5576d937cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:52.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.053+0000 7fd412ff5700 10 client.14934 ms_handle_connect on v2:172.21.15.204:6834/3870850287 2024-05-03T20:35:52.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.053+0000 7fd42effd700 1 --2- 172.21.15.181:0/114214962 >> [v2:172.21.15.204:6834/3870850287,v1:172.21.15.204:6835/3870850287] conn(0x5576d937e2a0 0x5576d937cfb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dce6b8eeeb57352a server_cookie=f6a9009d482e952d in_seq=0 out_seq=0 2024-05-03T20:35:52.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.054+0000 7fd412ff5700 1 -- 172.21.15.181:0/114214962 <== mds.4 v2:172.21.15.204:6834/3870850287 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5576d924f290 con 0x5576d937e2a0 2024-05-03T20:35:52.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.054+0000 7fd412ff5700 10 client.14934 handle_command_reply: tid=0 2024-05-03T20:35:52.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.054+0000 7fd43fd1db80 1 client.14934 shutdown 2024-05-03T20:35:52.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.054+0000 7fd43fd1db80 1 -- 172.21.15.181:0/114214962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f5530 msgr2=0x5576d9318980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:52.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.054+0000 7fd43fd1db80 1 --2- 172.21.15.181:0/114214962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f5530 0x5576d9318980 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd424005d90 tx=0x7fd42401e7f0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.054+0000 7fd43fd1db80 1 -- 172.21.15.181:0/114214962 shutdown_connections 2024-05-03T20:35:52.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.054+0000 7fd43fd1db80 1 --2- 172.21.15.181:0/114214962 >> [v2:172.21.15.204:6834/3870850287,v1:172.21.15.204:6835/3870850287] conn(0x5576d937e2a0 0x5576d937cfb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.054+0000 7fd43fd1db80 1 --2- 172.21.15.181:0/114214962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d92f9230 0x5576d9318ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.054+0000 7fd43fd1db80 1 --2- 172.21.15.181:0/114214962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5576d92f4c10 0x5576d92fb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.054+0000 7fd43fd1db80 1 --2- 172.21.15.181:0/114214962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5576d92f5530 0x5576d9318980 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.054+0000 7fd412ff5700 0 client.14934 ms_handle_reset on v2:172.21.15.204:6834/3870850287 2024-05-03T20:35:52.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.054+0000 7fd43fd1db80 1 -- 172.21.15.181:0/114214962 >> 172.21.15.181:0/114214962 conn(0x5576d92210b0 msgr2=0x5576d92f3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:52.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.054+0000 7fd43fd1db80 1 -- 172.21.15.181:0/114214962 shutdown_connections 2024-05-03T20:35:52.060 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.054+0000 7fd43fd1db80 1 -- 172.21.15.181:0/114214962 wait complete. 2024-05-03T20:35:52.060 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.054+0000 7fd43fd1db80 20 client.14934 trim_cache size 0 max 16384 2024-05-03T20:35:52.060 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-05-03T20:35:52.060 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "1cd058ee-098b-11ef-bc97-c7b262605968", 2024-05-03T20:35:52.060 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 4, 2024-05-03T20:35:52.060 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 24457, 2024-05-03T20:35:52.060 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2024-05-03T20:35:52.060 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2024-05-03T20:35:52.060 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2024-05-03T20:35:52.060 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 114.57820559300001, 2024-05-03T20:35:52.060 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 27, 2024-05-03T20:35:52.061 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 83, 2024-05-03T20:35:52.061 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 81, 2024-05-03T20:35:52.061 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 161.94061123399999 2024-05-03T20:35:52.061 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-05-03T20:35:52.061 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.055+0000 7fd43548a700 1 -- 172.21.15.181:0/3067824757 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd4180b2910 msgr2=0x7fd4180b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:52.061 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.055+0000 7fd43548a700 1 --2- 172.21.15.181:0/3067824757 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd4180b2910 0x7fd4180b4dd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd4301030a0 tx=0x7fd42400b410 comp rx=0 tx=0).stop 2024-05-03T20:35:52.062 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.055+0000 7fd43548a700 1 -- 172.21.15.181:0/3067824757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd430103750 msgr2=0x7fd43019f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:52.062 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.055+0000 7fd43548a700 1 --2- 172.21.15.181:0/3067824757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd430103750 0x7fd43019f760 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd42000b580 tx=0x7fd4200047b0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.062 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.055+0000 7fd43548a700 1 -- 172.21.15.181:0/3067824757 shutdown_connections 2024-05-03T20:35:52.062 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.055+0000 7fd43548a700 1 --2- 172.21.15.181:0/3067824757 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd4180b2910 0x7fd4180b4dd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.062 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.055+0000 7fd43548a700 1 --2- 172.21.15.181:0/3067824757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd430108850 0x7fd430199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.062 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.055+0000 7fd43548a700 1 --2- 172.21.15.181:0/3067824757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd430103750 0x7fd43019f760 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.062 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.055+0000 7fd43548a700 1 --2- 172.21.15.181:0/3067824757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd430104100 0x7fd43019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:52.062 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.055+0000 7fd43548a700 1 -- 172.21.15.181:0/3067824757 >> 172.21.15.181:0/3067824757 conn(0x7fd43007a600 msgr2=0x7fd430101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:52.062 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.055+0000 7fd43548a700 1 -- 172.21.15.181:0/3067824757 shutdown_connections 2024-05-03T20:35:52.062 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:52.055+0000 7fd43548a700 1 -- 172.21.15.181:0/3067824757 wait complete. 2024-05-03T20:35:52.807 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-03T20:35:53.047 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:35:53.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:53 smithi196 ceph-mon[32858]: pgmap v352: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:35:53.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:53 smithi204 ceph-mon[33595]: pgmap v352: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:35:53.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:53 smithi181 ceph-mon[29801]: pgmap v352: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:35:54.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.675+0000 7f8d4a424700 1 -- 172.21.15.181:0/4159079265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d44102fe0 msgr2=0x7f8d44106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:54.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.675+0000 7f8d4a424700 1 --2- 172.21.15.181:0/4159079265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d44102fe0 0x7f8d44106fe0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8d380099c0 tx=0x7f8d38009cd0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.675+0000 7f8d4a424700 1 -- 172.21.15.181:0/4159079265 shutdown_connections 2024-05-03T20:35:54.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.675+0000 7f8d4a424700 1 --2- 172.21.15.181:0/4159079265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d441076c0 0x7f8d4410f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.675+0000 7f8d4a424700 1 --2- 172.21.15.181:0/4159079265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d44102630 0x7f8d44102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.675+0000 7f8d4a424700 1 --2- 172.21.15.181:0/4159079265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d44102fe0 0x7f8d44106fe0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.675+0000 7f8d4a424700 1 -- 172.21.15.181:0/4159079265 >> 172.21.15.181:0/4159079265 conn(0x7f8d44079370 msgr2=0x7f8d44079780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:54.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.675+0000 7f8d4a424700 1 -- 172.21.15.181:0/4159079265 shutdown_connections 2024-05-03T20:35:54.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.675+0000 7f8d4a424700 1 -- 172.21.15.181:0/4159079265 wait complete. 2024-05-03T20:35:54.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.675+0000 7f8d4a424700 1 Processor -- start 2024-05-03T20:35:54.675 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.676+0000 7f8d4a424700 1 -- start start 2024-05-03T20:35:54.676 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.676+0000 7f8d4a424700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d44102630 0x7f8d44104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:54.676 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.676+0000 7f8d4a424700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d44102fe0 0x7f8d441045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:54.676 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.676+0000 7f8d4a424700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d441076c0 0x7f8d44104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:54.676 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.676+0000 7f8d4a424700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8d44112550 con 0x7f8d44102630 2024-05-03T20:35:54.676 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.676+0000 7f8d4a424700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8d441123d0 con 0x7f8d441076c0 2024-05-03T20:35:54.676 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.676+0000 7f8d4a424700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8d441126d0 con 0x7f8d44102fe0 2024-05-03T20:35:54.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.676+0000 7f8d43fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d44102630 0x7f8d44104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:54.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.676+0000 7f8d43fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d44102630 0x7f8d44104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42720/0 (socket says 172.21.15.181:42720) 2024-05-03T20:35:54.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.676+0000 7f8d43fff700 1 -- 172.21.15.181:0/3717137748 learned_addr learned my addr 172.21.15.181:0/3717137748 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:54.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.676+0000 7f8d437fe700 1 --2- 172.21.15.181:0/3717137748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d44102fe0 0x7f8d441045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:54.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.676+0000 7f8d489c1700 1 --2- 172.21.15.181:0/3717137748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d441076c0 0x7f8d44104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:54.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.676+0000 7f8d43fff700 1 -- 172.21.15.181:0/3717137748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d44102fe0 msgr2=0x7f8d441045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:54.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.676+0000 7f8d43fff700 1 --2- 172.21.15.181:0/3717137748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d44102fe0 0x7f8d441045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.676+0000 7f8d43fff700 1 -- 172.21.15.181:0/3717137748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d441076c0 msgr2=0x7f8d44104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:54.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.676+0000 7f8d43fff700 1 --2- 172.21.15.181:0/3717137748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d441076c0 0x7f8d44104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.676+0000 7f8d43fff700 1 -- 172.21.15.181:0/3717137748 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d441a80f0 con 0x7f8d44102630 2024-05-03T20:35:54.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.676+0000 7f8d489c1700 1 --2- 172.21.15.181:0/3717137748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d441076c0 0x7f8d44104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:35:54.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.677+0000 7f8d43fff700 1 --2- 172.21.15.181:0/3717137748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d44102630 0x7f8d44104080 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400c920 tx=0x7f8d3400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:54.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.677+0000 7f8d417fa700 1 -- 172.21.15.181:0/3717137748 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34007b00 con 0x7f8d44102630 2024-05-03T20:35:54.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.677+0000 7f8d4a424700 1 -- 172.21.15.181:0/3717137748 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d441a83e0 con 0x7f8d44102630 2024-05-03T20:35:54.678 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.677+0000 7f8d417fa700 1 -- 172.21.15.181:0/3717137748 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d34004ce0 con 0x7f8d44102630 2024-05-03T20:35:54.678 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.677+0000 7f8d417fa700 1 -- 172.21.15.181:0/3717137748 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34005500 con 0x7f8d44102630 2024-05-03T20:35:54.678 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.677+0000 7f8d4a424700 1 -- 172.21.15.181:0/3717137748 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d441a88f0 con 0x7f8d44102630 2024-05-03T20:35:54.683 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.679+0000 7f8d417fa700 1 -- 172.21.15.181:0/3717137748 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f8d34007ca0 con 0x7f8d44102630 2024-05-03T20:35:54.683 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.680+0000 7f8d417fa700 1 --2- 172.21.15.181:0/3717137748 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8d2c0b2910 0x7f8d2c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:54.683 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.680+0000 7f8d437fe700 1 --2- 172.21.15.181:0/3717137748 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8d2c0b2910 0x7f8d2c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:54.683 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.680+0000 7f8d417fa700 1 -- 172.21.15.181:0/3717137748 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f8d340e1f40 con 0x7f8d44102630 2024-05-03T20:35:54.683 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.680+0000 7f8d437fe700 1 --2- 172.21.15.181:0/3717137748 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8d2c0b2910 0x7f8d2c0b4dd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8d38006010 tx=0x7f8d38009630 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:54.683 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.680+0000 7f8d54cb7b80 1 Processor -- start 2024-05-03T20:35:54.683 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.680+0000 7f8d54cb7b80 1 -- start start 2024-05-03T20:35:54.683 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.681+0000 7f8d54cb7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b00e67740 0x561b00e67b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:54.683 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.681+0000 7f8d54cb7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b00e68060 0x561b00e62f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:54.684 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.681+0000 7f8d54cb7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b00e63480 0x561b00e638c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:54.684 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.681+0000 7f8d54cb7b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x561b00db58f0 con 0x561b00e63480 2024-05-03T20:35:54.684 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.681+0000 7f8d54cb7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561b00d1ad50 con 0x561b00e67740 2024-05-03T20:35:54.684 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.681+0000 7f8d54cb7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561b00db18b0 con 0x561b00e68060 2024-05-03T20:35:54.684 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.681+0000 7f8d489c1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b00e63480 0x561b00e638c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:54.684 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.681+0000 7f8d489c1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b00e63480 0x561b00e638c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42730/0 (socket says 172.21.15.181:42730) 2024-05-03T20:35:54.684 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.681+0000 7f8d489c1700 1 -- 172.21.15.181:0/2463000019 learned_addr learned my addr 172.21.15.181:0/2463000019 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:54.684 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.681+0000 7f8d43fff700 1 --2- 172.21.15.181:0/2463000019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b00e67740 0x561b00e67b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:54.684 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.681+0000 7f8d437fe700 1 --2- 172.21.15.181:0/2463000019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b00e68060 0x561b00e62f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:54.684 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.681+0000 7f8d489c1700 1 -- 172.21.15.181:0/2463000019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b00e68060 msgr2=0x561b00e62f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:35:54.684 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.681+0000 7f8d489c1700 1 --2- 172.21.15.181:0/2463000019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b00e68060 0x561b00e62f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.684 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.681+0000 7f8d489c1700 1 -- 172.21.15.181:0/2463000019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b00e67740 msgr2=0x561b00e67b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:35:54.684 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.681+0000 7f8d489c1700 1 --2- 172.21.15.181:0/2463000019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b00e67740 0x561b00e67b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.684 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.681+0000 7f8d489c1700 1 -- 172.21.15.181:0/2463000019 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b00d02880 con 0x561b00e63480 2024-05-03T20:35:54.684 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.681+0000 7f8d43fff700 1 --2- 172.21.15.181:0/2463000019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b00e67740 0x561b00e67b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:35:54.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.681+0000 7f8d489c1700 1 --2- 172.21.15.181:0/2463000019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b00e63480 0x561b00e638c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0085a0 tx=0x7f8d3c0088b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38ef60f2f40266b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:54.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.681+0000 7f8d2a7fc700 1 -- 172.21.15.181:0/2463000019 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c003c00 con 0x561b00e63480 2024-05-03T20:35:54.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.681+0000 7f8d2a7fc700 1 -- 172.21.15.181:0/2463000019 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d3c003da0 con 0x561b00e63480 2024-05-03T20:35:54.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.681+0000 7f8d2a7fc700 1 -- 172.21.15.181:0/2463000019 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c0129c0 con 0x561b00e63480 2024-05-03T20:35:54.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.682+0000 7f8d54cb7b80 1 -- 172.21.15.181:0/2463000019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b00e63480 msgr2=0x561b00e638c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:54.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.682+0000 7f8d54cb7b80 1 --2- 172.21.15.181:0/2463000019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b00e63480 0x561b00e638c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0085a0 tx=0x7f8d3c0088b0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.682+0000 7f8d54cb7b80 1 -- 172.21.15.181:0/2463000019 shutdown_connections 2024-05-03T20:35:54.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.682+0000 7f8d54cb7b80 1 --2- 172.21.15.181:0/2463000019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b00e63480 0x561b00e638c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.682+0000 7f8d54cb7b80 1 --2- 172.21.15.181:0/2463000019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b00e67740 0x561b00e67b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.682+0000 7f8d54cb7b80 1 --2- 172.21.15.181:0/2463000019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b00e68060 0x561b00e62f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.682+0000 7f8d54cb7b80 1 -- 172.21.15.181:0/2463000019 >> 172.21.15.181:0/2463000019 conn(0x561b00d8fb20 msgr2=0x561b00e5d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:54.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.682+0000 7f8d54cb7b80 1 -- 172.21.15.181:0/2463000019 shutdown_connections 2024-05-03T20:35:54.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.682+0000 7f8d54cb7b80 1 -- 172.21.15.181:0/2463000019 wait complete. 2024-05-03T20:35:54.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.682+0000 7f8d54cb7b80 1 -- start start 2024-05-03T20:35:54.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.683+0000 7f8d54cb7b80 1 Processor -- start 2024-05-03T20:35:54.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.683+0000 7f8d54cb7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b00e63480 0x561b00e6ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:54.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.683+0000 7f8d54cb7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b00e67740 0x561b00e6c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:54.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.683+0000 7f8d54cb7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b00e68060 0x561b00e6c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:54.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.683+0000 7f8d54cb7b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x561b00e6fba0 con 0x561b00e63480 2024-05-03T20:35:54.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.683+0000 7f8d54cb7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561b00d6ee80 con 0x561b00e67740 2024-05-03T20:35:54.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.683+0000 7f8d54cb7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561b00e6fd20 con 0x561b00e68060 2024-05-03T20:35:54.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.683+0000 7f8d54cb7b80 10 client.0 _ll_register_callbacks cb 0x561b00d49560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T20:35:54.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.683+0000 7f8d43fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b00e63480 0x561b00e6ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:54.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.683+0000 7f8d43fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b00e63480 0x561b00e6ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42740/0 (socket says 172.21.15.181:42740) 2024-05-03T20:35:54.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.683+0000 7f8d43fff700 1 -- 172.21.15.181:0/3984879462 learned_addr learned my addr 172.21.15.181:0/3984879462 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:54.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.683+0000 7f8d489c1700 1 --2- 172.21.15.181:0/3984879462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b00e68060 0x561b00e6c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:54.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.683+0000 7f8d437fe700 1 --2- 172.21.15.181:0/3984879462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b00e67740 0x561b00e6c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:54.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.683+0000 7f8d43fff700 1 -- 172.21.15.181:0/3984879462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b00e68060 msgr2=0x561b00e6c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:35:54.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.683+0000 7f8d43fff700 1 --2- 172.21.15.181:0/3984879462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b00e68060 0x561b00e6c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.683+0000 7f8d43fff700 1 -- 172.21.15.181:0/3984879462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b00e67740 msgr2=0x561b00e6c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:35:54.687 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.683+0000 7f8d43fff700 1 --2- 172.21.15.181:0/3984879462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b00e67740 0x561b00e6c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.687 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.683+0000 7f8d43fff700 1 -- 172.21.15.181:0/3984879462 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b00d6c710 con 0x561b00e63480 2024-05-03T20:35:54.687 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.683+0000 7f8d437fe700 1 --2- 172.21.15.181:0/3984879462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b00e67740 0x561b00e6c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:35:54.687 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.684+0000 7f8d43fff700 1 --2- 172.21.15.181:0/3984879462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b00e63480 0x561b00e6ed20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8d3401e000 tx=0x7f8d340a98e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a20fbc54e5ee6159 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:54.687 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.684+0000 7f8d237fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-03T20:35:54.687 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.684+0000 7f8d237fe700 1 -- 172.21.15.181:0/3984879462 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3401d040 con 0x561b00e63480 2024-05-03T20:35:54.687 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.684+0000 7f8d237fe700 1 -- 172.21.15.181:0/3984879462 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f8d340056c0 con 0x561b00e63480 2024-05-03T20:35:54.687 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.684+0000 7f8d54cb7b80 1 -- 172.21.15.181:0/3984879462 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d3c003da0 con 0x561b00e63480 2024-05-03T20:35:54.687 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.684+0000 7f8d237fe700 1 -- 172.21.15.181:0/3984879462 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8d340a9b00 con 0x561b00e63480 2024-05-03T20:35:54.687 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.684+0000 7f8d54cb7b80 10 client.14946 fetch_fsmap learned FSMap version 27 2024-05-03T20:35:54.687 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.684+0000 7f8d54cb7b80 1 -- 172.21.15.181:0/3984879462 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561b00e7a2b0 con 0x561b00e63480 2024-05-03T20:35:54.687 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.684+0000 7f8d237fe700 1 -- 172.21.15.181:0/3984879462 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d3c003da0 con 0x561b00e63480 2024-05-03T20:35:54.687 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.684+0000 7f8d237fe700 1 -- 172.21.15.181:0/3984879462 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8d340a3b20 con 0x561b00e63480 2024-05-03T20:35:54.687 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.684+0000 7f8d54cb7b80 10 client.14946 fetch_fsmap finished waiting for FSMap version 27 2024-05-03T20:35:54.687 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.684+0000 7f8d54cb7b80 10 client.14946 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-03T20:35:54.687 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.684+0000 7f8d54cb7b80 20 client.14946 populate_metadata read hostname 'smithi181' 2024-05-03T20:35:54.687 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.684+0000 7f8d54cb7b80 1 --2- 172.21.15.181:0/3984879462 >> [v2:172.21.15.204:6836/3006728559,v1:172.21.15.204:6837/3006728559] conn(0x561b00e7c830 0x561b00e7dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:54.688 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.684+0000 7f8d54cb7b80 4 client.14946 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T20:35:54.688 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.684+0000 7f8d54cb7b80 1 -- 172.21.15.181:0/3984879462 --> [v2:172.21.15.204:6836/3006728559,v1:172.21.15.204:6837/3006728559] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b00dc32d0 con 0x561b00e7c830 2024-05-03T20:35:54.688 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.685+0000 7f8d437fe700 1 --2- 172.21.15.181:0/3984879462 >> [v2:172.21.15.204:6836/3006728559,v1:172.21.15.204:6837/3006728559] conn(0x561b00e7c830 0x561b00e7dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:54.689 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.685+0000 7f8d237fe700 10 client.14946 ms_handle_connect on v2:172.21.15.204:6836/3006728559 2024-05-03T20:35:54.689 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.685+0000 7f8d437fe700 1 --2- 172.21.15.181:0/3984879462 >> [v2:172.21.15.204:6836/3006728559,v1:172.21.15.204:6837/3006728559] conn(0x561b00e7c830 0x561b00e7dca0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3763e1ca1f6a5e81 server_cookie=9804ff3d0010661c in_seq=0 out_seq=0 2024-05-03T20:35:54.689 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.686+0000 7f8d237fe700 1 -- 172.21.15.181:0/3984879462 <== mds.3 v2:172.21.15.204:6836/3006728559 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561b00dc32d0 con 0x561b00e7c830 2024-05-03T20:35:54.689 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.686+0000 7f8d237fe700 10 client.14946 handle_command_reply: tid=0 2024-05-03T20:35:54.689 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.686+0000 7f8d54cb7b80 1 client.14946 shutdown 2024-05-03T20:35:54.689 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.686+0000 7f8d54cb7b80 1 -- 172.21.15.181:0/3984879462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b00e63480 msgr2=0x561b00e6ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:54.689 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.686+0000 7f8d54cb7b80 1 --2- 172.21.15.181:0/3984879462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b00e63480 0x561b00e6ed20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8d3401e000 tx=0x7f8d340a98e0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.689 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.686+0000 7f8d54cb7b80 1 -- 172.21.15.181:0/3984879462 shutdown_connections 2024-05-03T20:35:54.689 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.686+0000 7f8d54cb7b80 1 --2- 172.21.15.181:0/3984879462 >> [v2:172.21.15.204:6836/3006728559,v1:172.21.15.204:6837/3006728559] conn(0x561b00e7c830 0x561b00e7dca0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.689 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.686+0000 7f8d54cb7b80 1 --2- 172.21.15.181:0/3984879462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b00e68060 0x561b00e6c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.689 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.686+0000 7f8d54cb7b80 1 --2- 172.21.15.181:0/3984879462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b00e63480 0x561b00e6ed20 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.689 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.686+0000 7f8d237fe700 0 client.14946 ms_handle_reset on v2:172.21.15.204:6836/3006728559 2024-05-03T20:35:54.689 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.686+0000 7f8d54cb7b80 1 --2- 172.21.15.181:0/3984879462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b00e67740 0x561b00e6c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.690 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.686+0000 7f8d54cb7b80 1 -- 172.21.15.181:0/3984879462 >> 172.21.15.181:0/3984879462 conn(0x561b00d8fb20 msgr2=0x561b00e68d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:54.690 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.686+0000 7f8d54cb7b80 1 -- 172.21.15.181:0/3984879462 shutdown_connections 2024-05-03T20:35:54.690 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.686+0000 7f8d54cb7b80 1 -- 172.21.15.181:0/3984879462 wait complete. 2024-05-03T20:35:54.690 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.686+0000 7f8d54cb7b80 20 client.14946 trim_cache size 0 max 16384 2024-05-03T20:35:54.706 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.707+0000 7f8d54cb7b80 1 Processor -- start 2024-05-03T20:35:54.706 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.707+0000 7f8d54cb7b80 1 -- start start 2024-05-03T20:35:54.707 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.707+0000 7f8d54cb7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b00e63480 0x561b00e77fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:54.707 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.707+0000 7f8d54cb7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b00e67740 0x561b00e784f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:54.707 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.707+0000 7f8d54cb7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b00e68060 0x561b00e78a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:54.707 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.707+0000 7f8d54cb7b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x561b00e7bfe0 con 0x561b00e68060 2024-05-03T20:35:54.707 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.707+0000 7f8d54cb7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561b00e7c6b0 con 0x561b00e63480 2024-05-03T20:35:54.707 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.707+0000 7f8d54cb7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561b00e7cd80 con 0x561b00e67740 2024-05-03T20:35:54.707 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.708+0000 7f8d43fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b00e63480 0x561b00e77fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:54.707 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.708+0000 7f8d43fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b00e63480 0x561b00e77fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.181:38006/0 (socket says 172.21.15.181:38006) 2024-05-03T20:35:54.707 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.708+0000 7f8d489c1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b00e68060 0x561b00e78a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:54.707 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.708+0000 7f8d43fff700 1 -- 172.21.15.181:0/3132750938 learned_addr learned my addr 172.21.15.181:0/3132750938 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:54.707 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.708+0000 7f8d437fe700 1 --2- 172.21.15.181:0/3132750938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b00e67740 0x561b00e784f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:54.707 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.708+0000 7f8d43fff700 1 -- 172.21.15.181:0/3132750938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b00e67740 msgr2=0x561b00e784f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:35:54.707 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.708+0000 7f8d43fff700 1 --2- 172.21.15.181:0/3132750938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b00e67740 0x561b00e784f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.707 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.708+0000 7f8d43fff700 1 -- 172.21.15.181:0/3132750938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b00e68060 msgr2=0x561b00e78a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:35:54.707 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.708+0000 7f8d43fff700 1 --2- 172.21.15.181:0/3132750938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b00e68060 0x561b00e78a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.707 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.708+0000 7f8d43fff700 1 -- 172.21.15.181:0/3132750938 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b00e67390 con 0x561b00e63480 2024-05-03T20:35:54.708 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.708+0000 7f8d489c1700 1 --2- 172.21.15.181:0/3132750938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b00e68060 0x561b00e78a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T20:35:54.708 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.708+0000 7f8d43fff700 1 --2- 172.21.15.181:0/3132750938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b00e63480 0x561b00e77fb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8d340a59a0 tx=0x7f8d34000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe3da10c873e0baf server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:54.708 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.708+0000 7f8d29ffb700 1 -- 172.21.15.181:0/3132750938 <== mon.2 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d340a50c0 con 0x561b00e63480 2024-05-03T20:35:54.708 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.708+0000 7f8d29ffb700 1 -- 172.21.15.181:0/3132750938 <== mon.2 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d340a9400 con 0x561b00e63480 2024-05-03T20:35:54.708 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.708+0000 7f8d29ffb700 1 -- 172.21.15.181:0/3132750938 <== mon.2 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d340a3b20 con 0x561b00e63480 2024-05-03T20:35:54.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.709+0000 7f8d54cb7b80 1 -- 172.21.15.181:0/3132750938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b00e63480 msgr2=0x561b00e77fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:54.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.709+0000 7f8d54cb7b80 1 --2- 172.21.15.181:0/3132750938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b00e63480 0x561b00e77fb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8d340a59a0 tx=0x7f8d34000bc0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.709+0000 7f8d54cb7b80 1 -- 172.21.15.181:0/3132750938 shutdown_connections 2024-05-03T20:35:54.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.709+0000 7f8d54cb7b80 1 --2- 172.21.15.181:0/3132750938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b00e68060 0x561b00e78a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.709+0000 7f8d54cb7b80 1 --2- 172.21.15.181:0/3132750938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b00e63480 0x561b00e77fb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.709+0000 7f8d54cb7b80 1 --2- 172.21.15.181:0/3132750938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b00e67740 0x561b00e784f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.709+0000 7f8d54cb7b80 1 -- 172.21.15.181:0/3132750938 >> 172.21.15.181:0/3132750938 conn(0x561b00d8fb20 msgr2=0x561b00e5c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:54.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.709+0000 7f8d54cb7b80 1 -- 172.21.15.181:0/3132750938 shutdown_connections 2024-05-03T20:35:54.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.709+0000 7f8d54cb7b80 1 -- 172.21.15.181:0/3132750938 wait complete. 2024-05-03T20:35:54.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.709+0000 7f8d54cb7b80 1 -- start start 2024-05-03T20:35:54.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.710+0000 7f8d54cb7b80 1 Processor -- start 2024-05-03T20:35:54.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.710+0000 7f8d54cb7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b00e63480 0x561b00e86110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:54.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.710+0000 7f8d54cb7b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b00e67740 0x561b00e86650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:54.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.710+0000 7f8d54cb7b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b00e68060 0x561b00e86b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:54.713 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.710+0000 7f8d54cb7b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x561b00e7db20 con 0x561b00e68060 2024-05-03T20:35:54.713 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.710+0000 7f8d54cb7b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561b00e7d450 con 0x561b00e67740 2024-05-03T20:35:54.713 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.710+0000 7f8d54cb7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561b00e7e1f0 con 0x561b00e63480 2024-05-03T20:35:54.713 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.710+0000 7f8d54cb7b80 10 client.0 _ll_register_callbacks cb 0x561b00d49560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T20:35:54.713 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.710+0000 7f8d489c1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b00e68060 0x561b00e86b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:54.713 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.710+0000 7f8d489c1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b00e68060 0x561b00e86b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42764/0 (socket says 172.21.15.181:42764) 2024-05-03T20:35:54.713 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.710+0000 7f8d489c1700 1 -- 172.21.15.181:0/710066822 learned_addr learned my addr 172.21.15.181:0/710066822 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:54.713 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.710+0000 7f8d43fff700 1 --2- 172.21.15.181:0/710066822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b00e63480 0x561b00e86110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:54.713 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.710+0000 7f8d437fe700 1 --2- 172.21.15.181:0/710066822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b00e67740 0x561b00e86650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:54.713 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.710+0000 7f8d489c1700 1 -- 172.21.15.181:0/710066822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b00e63480 msgr2=0x561b00e86110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:35:54.713 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.710+0000 7f8d489c1700 1 --2- 172.21.15.181:0/710066822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b00e63480 0x561b00e86110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.713 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.710+0000 7f8d489c1700 1 -- 172.21.15.181:0/710066822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b00e67740 msgr2=0x561b00e86650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:35:54.713 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.710+0000 7f8d489c1700 1 --2- 172.21.15.181:0/710066822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b00e67740 0x561b00e86650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.713 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.710+0000 7f8d489c1700 1 -- 172.21.15.181:0/710066822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b00e873d0 con 0x561b00e68060 2024-05-03T20:35:54.713 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.710+0000 7f8d437fe700 1 --2- 172.21.15.181:0/710066822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b00e67740 0x561b00e86650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:35:54.713 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.711+0000 7f8d489c1700 1 --2- 172.21.15.181:0/710066822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b00e68060 0x561b00e86b90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c008570 tx=0x7f8d3c012840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f67ea30e0e17917 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:54.713 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.711+0000 7f8d237fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-03T20:35:54.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.711+0000 7f8d237fe700 1 -- 172.21.15.181:0/710066822 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c024030 con 0x561b00e68060 2024-05-03T20:35:54.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.711+0000 7f8d54cb7b80 1 -- 172.21.15.181:0/710066822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d340a9400 con 0x561b00e68060 2024-05-03T20:35:54.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.711+0000 7f8d237fe700 1 -- 172.21.15.181:0/710066822 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f8d3c01a040 con 0x561b00e68060 2024-05-03T20:35:54.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.711+0000 7f8d237fe700 1 -- 172.21.15.181:0/710066822 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8d3c012c70 con 0x561b00e68060 2024-05-03T20:35:54.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.711+0000 7f8d54cb7b80 10 client.14952 fetch_fsmap learned FSMap version 27 2024-05-03T20:35:54.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.711+0000 7f8d54cb7b80 1 -- 172.21.15.181:0/710066822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561b00e68d20 con 0x561b00e68060 2024-05-03T20:35:54.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.711+0000 7f8d237fe700 1 -- 172.21.15.181:0/710066822 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d340a9400 con 0x561b00e68060 2024-05-03T20:35:54.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.711+0000 7f8d237fe700 1 -- 172.21.15.181:0/710066822 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8d3c013070 con 0x561b00e68060 2024-05-03T20:35:54.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.711+0000 7f8d54cb7b80 10 client.14952 fetch_fsmap finished waiting for FSMap version 27 2024-05-03T20:35:54.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.711+0000 7f8d54cb7b80 10 client.14952 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-03T20:35:54.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.711+0000 7f8d54cb7b80 20 client.14952 populate_metadata read hostname 'smithi181' 2024-05-03T20:35:54.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.711+0000 7f8d54cb7b80 1 --2- 172.21.15.181:0/710066822 >> [v2:172.21.15.204:6836/3006728559,v1:172.21.15.204:6837/3006728559] conn(0x561b00e7c830 0x561b00eebd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:54.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.711+0000 7f8d54cb7b80 4 client.14952 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T20:35:54.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.711+0000 7f8d54cb7b80 1 -- 172.21.15.181:0/710066822 --> [v2:172.21.15.204:6836/3006728559,v1:172.21.15.204:6837/3006728559] -- command(tid 0: {"prefix": "status"}) -- 0x561b00dbd370 con 0x561b00e7c830 2024-05-03T20:35:54.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.712+0000 7f8d43fff700 1 --2- 172.21.15.181:0/710066822 >> [v2:172.21.15.204:6836/3006728559,v1:172.21.15.204:6837/3006728559] conn(0x561b00e7c830 0x561b00eebd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.712+0000 7f8d43fff700 1 --2- 172.21.15.181:0/710066822 >> [v2:172.21.15.204:6836/3006728559,v1:172.21.15.204:6837/3006728559] conn(0x561b00e7c830 0x561b00eebd60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=64f44d2cb21dc429 server_cookie=1356edb692c56fbe in_seq=0 out_seq=0 2024-05-03T20:35:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.712+0000 7f8d237fe700 10 client.14952 ms_handle_connect on v2:172.21.15.204:6836/3006728559 2024-05-03T20:35:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.712+0000 7f8d237fe700 1 -- 172.21.15.181:0/710066822 <== mds.3 v2:172.21.15.204:6836/3006728559 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561b00dbd370 con 0x561b00e7c830 2024-05-03T20:35:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.712+0000 7f8d237fe700 10 client.14952 handle_command_reply: tid=0 2024-05-03T20:35:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.712+0000 7f8d54cb7b80 1 client.14952 shutdown 2024-05-03T20:35:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.713+0000 7f8d54cb7b80 1 -- 172.21.15.181:0/710066822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b00e68060 msgr2=0x561b00e86b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.713+0000 7f8d54cb7b80 1 --2- 172.21.15.181:0/710066822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b00e68060 0x561b00e86b90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c008570 tx=0x7f8d3c012840 comp rx=0 tx=0).stop 2024-05-03T20:35:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.713+0000 7f8d54cb7b80 1 -- 172.21.15.181:0/710066822 shutdown_connections 2024-05-03T20:35:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.713+0000 7f8d54cb7b80 1 --2- 172.21.15.181:0/710066822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b00e68060 0x561b00e86b90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.713+0000 7f8d54cb7b80 1 --2- 172.21.15.181:0/710066822 >> [v2:172.21.15.204:6836/3006728559,v1:172.21.15.204:6837/3006728559] conn(0x561b00e7c830 0x561b00eebd60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.713+0000 7f8d237fe700 0 client.14952 ms_handle_reset on v2:172.21.15.204:6836/3006728559 2024-05-03T20:35:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.713+0000 7f8d54cb7b80 1 --2- 172.21.15.181:0/710066822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b00e63480 0x561b00e86110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.713+0000 7f8d54cb7b80 1 --2- 172.21.15.181:0/710066822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b00e67740 0x561b00e86650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.713+0000 7f8d54cb7b80 1 -- 172.21.15.181:0/710066822 >> 172.21.15.181:0/710066822 conn(0x561b00d8fb20 msgr2=0x561b00e78840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.713+0000 7f8d54cb7b80 1 -- 172.21.15.181:0/710066822 shutdown_connections 2024-05-03T20:35:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.713+0000 7f8d54cb7b80 1 -- 172.21.15.181:0/710066822 wait complete. 2024-05-03T20:35:54.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.713+0000 7f8d54cb7b80 20 client.14952 trim_cache size 0 max 16384 2024-05-03T20:35:54.716 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-05-03T20:35:54.716 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "1cd058ee-098b-11ef-bc97-c7b262605968", 2024-05-03T20:35:54.716 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 3, 2024-05-03T20:35:54.716 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 24463, 2024-05-03T20:35:54.716 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2024-05-03T20:35:54.716 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2024-05-03T20:35:54.716 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2024-05-03T20:35:54.716 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 118.246628984, 2024-05-03T20:35:54.716 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 27, 2024-05-03T20:35:54.716 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 83, 2024-05-03T20:35:54.716 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 81, 2024-05-03T20:35:54.716 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 161.83512995800001 2024-05-03T20:35:54.717 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-05-03T20:35:54.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.713+0000 7f8d4a424700 1 -- 172.21.15.181:0/3717137748 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8d2c0b2910 msgr2=0x7f8d2c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:54.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.713+0000 7f8d4a424700 1 --2- 172.21.15.181:0/3717137748 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8d2c0b2910 0x7f8d2c0b4dd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8d38006010 tx=0x7f8d38009630 comp rx=0 tx=0).stop 2024-05-03T20:35:54.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.713+0000 7f8d4a424700 1 -- 172.21.15.181:0/3717137748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d44102630 msgr2=0x7f8d44104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:54.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.713+0000 7f8d4a424700 1 --2- 172.21.15.181:0/3717137748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d44102630 0x7f8d44104080 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400c920 tx=0x7f8d3400cc30 comp rx=0 tx=0).stop 2024-05-03T20:35:54.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.714+0000 7f8d4a424700 1 -- 172.21.15.181:0/3717137748 shutdown_connections 2024-05-03T20:35:54.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.714+0000 7f8d4a424700 1 --2- 172.21.15.181:0/3717137748 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8d2c0b2910 0x7f8d2c0b4dd0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.714+0000 7f8d4a424700 1 --2- 172.21.15.181:0/3717137748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d441076c0 0x7f8d44104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.714+0000 7f8d4a424700 1 --2- 172.21.15.181:0/3717137748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d44102630 0x7f8d44104080 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.714+0000 7f8d4a424700 1 --2- 172.21.15.181:0/3717137748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d44102fe0 0x7f8d441045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:54.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.714+0000 7f8d4a424700 1 -- 172.21.15.181:0/3717137748 >> 172.21.15.181:0/3717137748 conn(0x7f8d44079370 msgr2=0x7f8d441063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:54.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.714+0000 7f8d4a424700 1 -- 172.21.15.181:0/3717137748 shutdown_connections 2024-05-03T20:35:54.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:54.714+0000 7f8d4a424700 1 -- 172.21.15.181:0/3717137748 wait complete. 2024-05-03T20:35:55.387 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph tell mds.h status 2024-05-03T20:35:55.629 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:35:55.693 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:55 smithi181 ceph-mon[29801]: pgmap v353: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:35:55.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:55 smithi196 ceph-mon[32858]: pgmap v353: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:35:55.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:55 smithi204 ceph-mon[33595]: pgmap v353: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:35:57.357 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.356+0000 7f78b3200700 1 -- 172.21.15.181:0/375304103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78ac108850 msgr2=0x7f78ac1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:57.357 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.356+0000 7f78b3200700 1 --2- 172.21.15.181:0/375304103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78ac108850 0x7f78ac1103d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f78a400b0d0 tx=0x7f78a400b3e0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.356+0000 7f78b3200700 1 -- 172.21.15.181:0/375304103 shutdown_connections 2024-05-03T20:35:57.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.356+0000 7f78b3200700 1 --2- 172.21.15.181:0/375304103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78ac108850 0x7f78ac1103d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.356+0000 7f78b3200700 1 --2- 172.21.15.181:0/375304103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78ac103750 0x7f78ac103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.356+0000 7f78b3200700 1 --2- 172.21.15.181:0/375304103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78ac104100 0x7f78ac108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.356+0000 7f78b3200700 1 -- 172.21.15.181:0/375304103 >> 172.21.15.181:0/375304103 conn(0x7f78ac07a600 msgr2=0x7f78ac07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:57.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.356+0000 7f78b3200700 1 -- 172.21.15.181:0/375304103 shutdown_connections 2024-05-03T20:35:57.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.356+0000 7f78b3200700 1 -- 172.21.15.181:0/375304103 wait complete. 2024-05-03T20:35:57.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.356+0000 7f78b3200700 1 Processor -- start 2024-05-03T20:35:57.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.356+0000 7f78b3200700 1 -- start start 2024-05-03T20:35:57.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.357+0000 7f78b3200700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78ac103750 0x7f78ac19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:57.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.357+0000 7f78b3200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78ac104100 0x7f78ac19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:57.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.357+0000 7f78b3200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78ac108850 0x7f78ac1998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:57.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.357+0000 7f78b3200700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac113640 con 0x7f78ac103750 2024-05-03T20:35:57.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.357+0000 7f78b3200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac1134c0 con 0x7f78ac108850 2024-05-03T20:35:57.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.357+0000 7f78b3200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac1137c0 con 0x7f78ac104100 2024-05-03T20:35:57.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.357+0000 7f78b0f9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78ac103750 0x7f78ac19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:57.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.357+0000 7f78b0f9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78ac103750 0x7f78ac19f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58098/0 (socket says 172.21.15.181:58098) 2024-05-03T20:35:57.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.357+0000 7f78b0f9c700 1 -- 172.21.15.181:0/718017832 learned_addr learned my addr 172.21.15.181:0/718017832 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:57.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.357+0000 7f78abfff700 1 --2- 172.21.15.181:0/718017832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78ac104100 0x7f78ac19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:57.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.357+0000 7f78b179d700 1 --2- 172.21.15.181:0/718017832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78ac108850 0x7f78ac1998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:57.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.357+0000 7f78b0f9c700 1 -- 172.21.15.181:0/718017832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78ac104100 msgr2=0x7f78ac19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:57.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.357+0000 7f78b0f9c700 1 --2- 172.21.15.181:0/718017832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78ac104100 0x7f78ac19fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.357+0000 7f78b0f9c700 1 -- 172.21.15.181:0/718017832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78ac108850 msgr2=0x7f78ac1998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:57.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.357+0000 7f78b0f9c700 1 --2- 172.21.15.181:0/718017832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78ac108850 0x7f78ac1998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.357+0000 7f78b0f9c700 1 -- 172.21.15.181:0/718017832 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ac19a020 con 0x7f78ac103750 2024-05-03T20:35:57.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.357+0000 7f78b179d700 1 --2- 172.21.15.181:0/718017832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78ac108850 0x7f78ac1998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:35:57.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.357+0000 7f78b0f9c700 1 --2- 172.21.15.181:0/718017832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78ac103750 0x7f78ac19f700 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f78a000d980 tx=0x7f78a000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:57.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.357+0000 7f78a9ffb700 1 -- 172.21.15.181:0/718017832 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a0017070 con 0x7f78ac103750 2024-05-03T20:35:57.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.357+0000 7f78a9ffb700 1 -- 172.21.15.181:0/718017832 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78a0004830 con 0x7f78ac103750 2024-05-03T20:35:57.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.357+0000 7f78b3200700 1 -- 172.21.15.181:0/718017832 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78ac19a310 con 0x7f78ac103750 2024-05-03T20:35:57.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.357+0000 7f78a9ffb700 1 -- 172.21.15.181:0/718017832 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a0005440 con 0x7f78ac103750 2024-05-03T20:35:57.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.358+0000 7f78b3200700 1 -- 172.21.15.181:0/718017832 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78ac1a65d0 con 0x7f78ac103750 2024-05-03T20:35:57.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.360+0000 7f78a9ffb700 1 -- 172.21.15.181:0/718017832 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f78a00055e0 con 0x7f78ac103750 2024-05-03T20:35:57.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.360+0000 7f78a9ffb700 1 --2- 172.21.15.181:0/718017832 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f789c0b2910 0x7f789c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:57.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.361+0000 7f78abfff700 1 --2- 172.21.15.181:0/718017832 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f789c0b2910 0x7f789c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:57.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.361+0000 7f78a9ffb700 1 -- 172.21.15.181:0/718017832 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f78a00e2350 con 0x7f78ac103750 2024-05-03T20:35:57.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.361+0000 7f78abfff700 1 --2- 172.21.15.181:0/718017832 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f789c0b2910 0x7f789c0b4dd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f78ac1030a0 tx=0x7f7898005cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:57.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.361+0000 7f78bda93b80 1 Processor -- start 2024-05-03T20:35:57.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.361+0000 7f78bda93b80 1 -- start start 2024-05-03T20:35:57.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.361+0000 7f78bda93b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d3503d00 0x5642d35040e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:57.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.361+0000 7f78bda93b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d3504620 0x5642d34fe350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:57.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.361+0000 7f78bda93b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642d34fe8e0 0x5642d34fed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:57.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.361+0000 7f78bda93b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5642d344c8f0 con 0x5642d3504620 2024-05-03T20:35:57.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.361+0000 7f78bda93b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5642d33b1d50 con 0x5642d34fe8e0 2024-05-03T20:35:57.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.361+0000 7f78bda93b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5642d34488b0 con 0x5642d3503d00 2024-05-03T20:35:57.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.362+0000 7f78abfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d3504620 0x5642d34fe350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:57.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.362+0000 7f78abfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d3504620 0x5642d34fe350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58106/0 (socket says 172.21.15.181:58106) 2024-05-03T20:35:57.370 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.362+0000 7f78abfff700 1 -- 172.21.15.181:0/910238366 learned_addr learned my addr 172.21.15.181:0/910238366 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:57.370 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.362+0000 7f78b0f9c700 1 --2- 172.21.15.181:0/910238366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d3503d00 0x5642d35040e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:57.370 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.362+0000 7f78abfff700 1 -- 172.21.15.181:0/910238366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d3503d00 msgr2=0x5642d35040e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:35:57.370 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.362+0000 7f78abfff700 1 --2- 172.21.15.181:0/910238366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d3503d00 0x5642d35040e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.370 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.362+0000 7f78b179d700 1 --2- 172.21.15.181:0/910238366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642d34fe8e0 0x5642d34fed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:57.370 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.362+0000 7f78abfff700 1 -- 172.21.15.181:0/910238366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642d34fe8e0 msgr2=0x5642d34fed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:35:57.370 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.362+0000 7f78abfff700 1 --2- 172.21.15.181:0/910238366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642d34fe8e0 0x5642d34fed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.370 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.362+0000 7f78abfff700 1 -- 172.21.15.181:0/910238366 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642d3399880 con 0x5642d3504620 2024-05-03T20:35:57.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.362+0000 7f78abfff700 1 --2- 172.21.15.181:0/910238366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d3504620 0x5642d34fe350 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f789800dbb0 tx=0x7f7898010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=720f77ba62bec8a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:57.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.362+0000 7f7896ffd700 1 -- 172.21.15.181:0/910238366 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7898020070 con 0x5642d3504620 2024-05-03T20:35:57.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.362+0000 7f7896ffd700 1 -- 172.21.15.181:0/910238366 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7898005020 con 0x5642d3504620 2024-05-03T20:35:57.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.362+0000 7f7896ffd700 1 -- 172.21.15.181:0/910238366 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789800f980 con 0x5642d3504620 2024-05-03T20:35:57.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.363+0000 7f78bda93b80 1 -- 172.21.15.181:0/910238366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d3504620 msgr2=0x5642d34fe350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:57.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.363+0000 7f78bda93b80 1 --2- 172.21.15.181:0/910238366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d3504620 0x5642d34fe350 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f789800dbb0 tx=0x7f7898010000 comp rx=0 tx=0).stop 2024-05-03T20:35:57.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.363+0000 7f78bda93b80 1 -- 172.21.15.181:0/910238366 shutdown_connections 2024-05-03T20:35:57.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.363+0000 7f78bda93b80 1 --2- 172.21.15.181:0/910238366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642d34fe8e0 0x5642d34fed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.363+0000 7f78bda93b80 1 --2- 172.21.15.181:0/910238366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d3503d00 0x5642d35040e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.363+0000 7f78bda93b80 1 --2- 172.21.15.181:0/910238366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d3504620 0x5642d34fe350 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.363+0000 7f78bda93b80 1 -- 172.21.15.181:0/910238366 >> 172.21.15.181:0/910238366 conn(0x5642d3426b20 msgr2=0x5642d34f49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:57.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.363+0000 7f78bda93b80 1 -- 172.21.15.181:0/910238366 shutdown_connections 2024-05-03T20:35:57.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.363+0000 7f78bda93b80 1 -- 172.21.15.181:0/910238366 wait complete. 2024-05-03T20:35:57.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.363+0000 7f78bda93b80 1 -- start start 2024-05-03T20:35:57.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.363+0000 7f78bda93b80 1 Processor -- start 2024-05-03T20:35:57.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.364+0000 7f78bda93b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d34fe8e0 0x5642d350a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:57.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.364+0000 7f78bda93b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d3503d00 0x5642d3507640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:57.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.364+0000 7f78bda93b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642d3504620 0x5642d3507b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:57.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.364+0000 7f78bda93b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5642d350b050 con 0x5642d3503d00 2024-05-03T20:35:57.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.364+0000 7f78bda93b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5642d3405e80 con 0x5642d3504620 2024-05-03T20:35:57.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.364+0000 7f78bda93b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5642d350b1d0 con 0x5642d34fe8e0 2024-05-03T20:35:57.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.364+0000 7f78abfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d3503d00 0x5642d3507640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:57.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.364+0000 7f78bda93b80 10 client.0 _ll_register_callbacks cb 0x5642d33e0560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T20:35:57.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.364+0000 7f78abfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d3503d00 0x5642d3507640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58108/0 (socket says 172.21.15.181:58108) 2024-05-03T20:35:57.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.364+0000 7f78abfff700 1 -- 172.21.15.181:0/1178742215 learned_addr learned my addr 172.21.15.181:0/1178742215 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:57.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.364+0000 7f78b0f9c700 1 --2- 172.21.15.181:0/1178742215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d34fe8e0 0x5642d350a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:57.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.364+0000 7f78abfff700 1 -- 172.21.15.181:0/1178742215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d34fe8e0 msgr2=0x5642d350a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:35:57.373 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.364+0000 7f78abfff700 1 --2- 172.21.15.181:0/1178742215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d34fe8e0 0x5642d350a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.373 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.364+0000 7f78abfff700 1 -- 172.21.15.181:0/1178742215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642d3504620 msgr2=0x5642d3507b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:35:57.373 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.364+0000 7f78abfff700 1 --2- 172.21.15.181:0/1178742215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642d3504620 0x5642d3507b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.373 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.364+0000 7f78abfff700 1 -- 172.21.15.181:0/1178742215 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642d3403710 con 0x5642d3503d00 2024-05-03T20:35:57.373 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.364+0000 7f78abfff700 1 --2- 172.21.15.181:0/1178742215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d3503d00 0x5642d3507640 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f789800db80 tx=0x7f7898004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b73fb84c93d27da server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:57.373 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.364+0000 7f788bfff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-03T20:35:57.373 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.365+0000 7f788bfff700 1 -- 172.21.15.181:0/1178742215 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7898020040 con 0x5642d3503d00 2024-05-03T20:35:57.373 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.365+0000 7f788bfff700 1 -- 172.21.15.181:0/1178742215 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f789800f6b0 con 0x5642d3503d00 2024-05-03T20:35:57.373 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.365+0000 7f78bda93b80 1 -- 172.21.15.181:0/1178742215 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7898005020 con 0x5642d3503d00 2024-05-03T20:35:57.373 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.365+0000 7f788bfff700 1 -- 172.21.15.181:0/1178742215 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f789800f8b0 con 0x5642d3503d00 2024-05-03T20:35:57.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.365+0000 7f78bda93b80 10 client.14964 fetch_fsmap learned FSMap version 27 2024-05-03T20:35:57.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.365+0000 7f78bda93b80 1 -- 172.21.15.181:0/1178742215 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5642d35158c0 con 0x5642d3503d00 2024-05-03T20:35:57.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.365+0000 7f788bfff700 1 -- 172.21.15.181:0/1178742215 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7898005020 con 0x5642d3503d00 2024-05-03T20:35:57.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.365+0000 7f788bfff700 1 -- 172.21.15.181:0/1178742215 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f789801ca00 con 0x5642d3503d00 2024-05-03T20:35:57.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.365+0000 7f78bda93b80 10 client.14964 fetch_fsmap finished waiting for FSMap version 27 2024-05-03T20:35:57.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.365+0000 7f78bda93b80 10 client.14964 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-03T20:35:57.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.365+0000 7f78bda93b80 20 client.14964 populate_metadata read hostname 'smithi181' 2024-05-03T20:35:57.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.365+0000 7f78bda93b80 1 --2- 172.21.15.181:0/1178742215 >> [v2:172.21.15.196:6836/1529420259,v1:172.21.15.196:6837/1529420259] conn(0x5642d3518b00 0x5642d35191d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:57.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.365+0000 7f78bda93b80 4 client.14964 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T20:35:57.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.365+0000 7f78bda93b80 1 -- 172.21.15.181:0/1178742215 --> [v2:172.21.15.196:6836/1529420259,v1:172.21.15.196:6837/1529420259] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642d345a2d0 con 0x5642d3518b00 2024-05-03T20:35:57.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.366+0000 7f78b0f9c700 1 --2- 172.21.15.181:0/1178742215 >> [v2:172.21.15.196:6836/1529420259,v1:172.21.15.196:6837/1529420259] conn(0x5642d3518b00 0x5642d35191d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:57.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.366+0000 7f78b0f9c700 1 --2- 172.21.15.181:0/1178742215 >> [v2:172.21.15.196:6836/1529420259,v1:172.21.15.196:6837/1529420259] conn(0x5642d3518b00 0x5642d35191d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=72e3c612443476f1 server_cookie=130b4fa74b08121b in_seq=0 out_seq=0 2024-05-03T20:35:57.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.366+0000 7f788bfff700 10 client.14964 ms_handle_connect on v2:172.21.15.196:6836/1529420259 2024-05-03T20:35:57.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.367+0000 7f788bfff700 1 -- 172.21.15.181:0/1178742215 <== mds.2 v2:172.21.15.196:6836/1529420259 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5642d345a2d0 con 0x5642d3518b00 2024-05-03T20:35:57.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.367+0000 7f788bfff700 10 client.14964 handle_command_reply: tid=0 2024-05-03T20:35:57.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.367+0000 7f78bda93b80 1 client.14964 shutdown 2024-05-03T20:35:57.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.367+0000 7f78bda93b80 1 -- 172.21.15.181:0/1178742215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d3503d00 msgr2=0x5642d3507640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:57.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.367+0000 7f78bda93b80 1 --2- 172.21.15.181:0/1178742215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d3503d00 0x5642d3507640 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f789800db80 tx=0x7f7898004a30 comp rx=0 tx=0).stop 2024-05-03T20:35:57.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.367+0000 7f78bda93b80 1 -- 172.21.15.181:0/1178742215 shutdown_connections 2024-05-03T20:35:57.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.367+0000 7f78bda93b80 1 --2- 172.21.15.181:0/1178742215 >> [v2:172.21.15.196:6836/1529420259,v1:172.21.15.196:6837/1529420259] conn(0x5642d3518b00 0x5642d35191d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.367+0000 7f78bda93b80 1 --2- 172.21.15.181:0/1178742215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642d3504620 0x5642d3507b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.367+0000 7f788bfff700 0 client.14964 ms_handle_reset on v2:172.21.15.196:6836/1529420259 2024-05-03T20:35:57.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.367+0000 7f78bda93b80 1 --2- 172.21.15.181:0/1178742215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d34fe8e0 0x5642d350a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.367+0000 7f78bda93b80 1 --2- 172.21.15.181:0/1178742215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d3503d00 0x5642d3507640 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.367+0000 7f78bda93b80 1 -- 172.21.15.181:0/1178742215 >> 172.21.15.181:0/1178742215 conn(0x5642d3426b20 msgr2=0x5642d35030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:57.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.367+0000 7f78bda93b80 1 -- 172.21.15.181:0/1178742215 shutdown_connections 2024-05-03T20:35:57.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.367+0000 7f78bda93b80 1 -- 172.21.15.181:0/1178742215 wait complete. 2024-05-03T20:35:57.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.367+0000 7f78bda93b80 20 client.14964 trim_cache size 0 max 16384 2024-05-03T20:35:57.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.388+0000 7f78bda93b80 1 Processor -- start 2024-05-03T20:35:57.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.388+0000 7f78bda93b80 1 -- start start 2024-05-03T20:35:57.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.388+0000 7f78bda93b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d34fe8e0 0x5642d34f4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:57.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.388+0000 7f78bda93b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642d3503d00 0x5642d3500790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:57.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.388+0000 7f78bda93b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d3504620 0x5642d3500cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:57.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.388+0000 7f78bda93b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5642d344c8f0 con 0x5642d34fe8e0 2024-05-03T20:35:57.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.388+0000 7f78bda93b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5642d3517e00 con 0x5642d3503d00 2024-05-03T20:35:57.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.388+0000 7f78bda93b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5642d35182b0 con 0x5642d3504620 2024-05-03T20:35:57.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.388+0000 7f78b0f9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d34fe8e0 0x5642d34f4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:57.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.388+0000 7f78b0f9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d34fe8e0 0x5642d34f4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58114/0 (socket says 172.21.15.181:58114) 2024-05-03T20:35:57.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.388+0000 7f78b0f9c700 1 -- 172.21.15.181:0/702426741 learned_addr learned my addr 172.21.15.181:0/702426741 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:57.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.389+0000 7f78abfff700 1 --2- 172.21.15.181:0/702426741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642d3503d00 0x5642d3500790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:57.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.389+0000 7f78b0f9c700 1 -- 172.21.15.181:0/702426741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d3504620 msgr2=0x5642d3500cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:35:57.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.389+0000 7f78b0f9c700 1 --2- 172.21.15.181:0/702426741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d3504620 0x5642d3500cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.389+0000 7f78b0f9c700 1 -- 172.21.15.181:0/702426741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642d3503d00 msgr2=0x5642d3500790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:35:57.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.389+0000 7f78b0f9c700 1 --2- 172.21.15.181:0/702426741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642d3503d00 0x5642d3500790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.389+0000 7f78b0f9c700 1 -- 172.21.15.181:0/702426741 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642d350b180 con 0x5642d34fe8e0 2024-05-03T20:35:57.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.389+0000 7f78abfff700 1 --2- 172.21.15.181:0/702426741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642d3503d00 0x5642d3500790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:35:57.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.389+0000 7f78b0f9c700 1 --2- 172.21.15.181:0/702426741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d34fe8e0 0x5642d34f4090 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f78a00a6a80 tx=0x7f78a0091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccbce35e56f67751 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:57.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.389+0000 7f78967fc700 1 -- 172.21.15.181:0/702426741 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a00b2670 con 0x5642d34fe8e0 2024-05-03T20:35:57.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.389+0000 7f78967fc700 1 -- 172.21.15.181:0/702426741 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78a00cc0b0 con 0x5642d34fe8e0 2024-05-03T20:35:57.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.389+0000 7f78967fc700 1 -- 172.21.15.181:0/702426741 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a00a9a70 con 0x5642d34fe8e0 2024-05-03T20:35:57.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.390+0000 7f78bda93b80 1 -- 172.21.15.181:0/702426741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d34fe8e0 msgr2=0x5642d34f4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:57.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.390+0000 7f78bda93b80 1 --2- 172.21.15.181:0/702426741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d34fe8e0 0x5642d34f4090 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f78a00a6a80 tx=0x7f78a0091040 comp rx=0 tx=0).stop 2024-05-03T20:35:57.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.390+0000 7f78bda93b80 1 -- 172.21.15.181:0/702426741 shutdown_connections 2024-05-03T20:35:57.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.390+0000 7f78bda93b80 1 --2- 172.21.15.181:0/702426741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d3504620 0x5642d3500cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.390+0000 7f78bda93b80 1 --2- 172.21.15.181:0/702426741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d34fe8e0 0x5642d34f4090 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.390+0000 7f78bda93b80 1 --2- 172.21.15.181:0/702426741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642d3503d00 0x5642d3500790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.390+0000 7f78bda93b80 1 -- 172.21.15.181:0/702426741 >> 172.21.15.181:0/702426741 conn(0x5642d3426b20 msgr2=0x5642d34f4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:57.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.390+0000 7f78bda93b80 1 -- 172.21.15.181:0/702426741 shutdown_connections 2024-05-03T20:35:57.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.390+0000 7f78bda93b80 1 -- 172.21.15.181:0/702426741 wait complete. 2024-05-03T20:35:57.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.390+0000 7f78bda93b80 1 -- start start 2024-05-03T20:35:57.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.390+0000 7f78bda93b80 1 Processor -- start 2024-05-03T20:35:57.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.390+0000 7f78bda93b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d34fe8e0 0x5642d34fff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:57.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.391+0000 7f78bda93b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642d3503d00 0x5642d35004d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:57.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.391+0000 7f78bda93b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d3504620 0x5642d3500a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:57.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.391+0000 7f78bda93b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5642d3519050 con 0x5642d34fe8e0 2024-05-03T20:35:57.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.391+0000 7f78bda93b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5642d3518980 con 0x5642d3503d00 2024-05-03T20:35:57.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.391+0000 7f78bda93b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5642d3519720 con 0x5642d3504620 2024-05-03T20:35:57.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.391+0000 7f78bda93b80 10 client.0 _ll_register_callbacks cb 0x5642d33e0560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T20:35:57.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.391+0000 7f78b0f9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d34fe8e0 0x5642d34fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:57.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.391+0000 7f78b0f9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d34fe8e0 0x5642d34fff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58128/0 (socket says 172.21.15.181:58128) 2024-05-03T20:35:57.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.391+0000 7f78b0f9c700 1 -- 172.21.15.181:0/945969808 learned_addr learned my addr 172.21.15.181:0/945969808 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:35:57.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.391+0000 7f78abfff700 1 --2- 172.21.15.181:0/945969808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642d3503d00 0x5642d35004d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:57.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.391+0000 7f78b179d700 1 --2- 172.21.15.181:0/945969808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d3504620 0x5642d3500a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:57.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.391+0000 7f78b0f9c700 1 -- 172.21.15.181:0/945969808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d3504620 msgr2=0x5642d3500a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:35:57.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.391+0000 7f78b0f9c700 1 --2- 172.21.15.181:0/945969808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d3504620 0x5642d3500a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.391+0000 7f78b0f9c700 1 -- 172.21.15.181:0/945969808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642d3503d00 msgr2=0x5642d35004d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:35:57.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.391+0000 7f78b0f9c700 1 --2- 172.21.15.181:0/945969808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642d3503d00 0x5642d35004d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.391+0000 7f78b0f9c700 1 -- 172.21.15.181:0/945969808 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642d3520de0 con 0x5642d34fe8e0 2024-05-03T20:35:57.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.391+0000 7f78abfff700 1 --2- 172.21.15.181:0/945969808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642d3503d00 0x5642d35004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:35:57.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.391+0000 7f78b0f9c700 1 --2- 172.21.15.181:0/945969808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d34fe8e0 0x5642d34fff90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f78a00a5890 tx=0x7f78a00096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05da940e8b1f1e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:35:57.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.391+0000 7f788bfff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-03T20:35:57.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.391+0000 7f788bfff700 1 -- 172.21.15.181:0/945969808 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a006f120 con 0x5642d34fe8e0 2024-05-03T20:35:57.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.391+0000 7f78bda93b80 1 -- 172.21.15.181:0/945969808 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78a00cc0b0 con 0x5642d34fe8e0 2024-05-03T20:35:57.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.391+0000 7f788bfff700 1 -- 172.21.15.181:0/945969808 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f78a00a9a70 con 0x5642d34fe8e0 2024-05-03T20:35:57.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.391+0000 7f788bfff700 1 -- 172.21.15.181:0/945969808 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f78a00a9c70 con 0x5642d34fe8e0 2024-05-03T20:35:57.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.392+0000 7f78bda93b80 10 client.14970 fetch_fsmap learned FSMap version 27 2024-05-03T20:35:57.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.392+0000 7f78bda93b80 1 -- 172.21.15.181:0/945969808 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5642d3583600 con 0x5642d34fe8e0 2024-05-03T20:35:57.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.392+0000 7f788bfff700 1 -- 172.21.15.181:0/945969808 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78a00cc0b0 con 0x5642d34fe8e0 2024-05-03T20:35:57.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.392+0000 7f788bfff700 1 -- 172.21.15.181:0/945969808 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f78a0009bd0 con 0x5642d34fe8e0 2024-05-03T20:35:57.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.392+0000 7f78bda93b80 10 client.14970 fetch_fsmap finished waiting for FSMap version 27 2024-05-03T20:35:57.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.392+0000 7f78bda93b80 10 client.14970 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-03T20:35:57.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.392+0000 7f78bda93b80 20 client.14970 populate_metadata read hostname 'smithi181' 2024-05-03T20:35:57.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.392+0000 7f78bda93b80 1 --2- 172.21.15.181:0/945969808 >> [v2:172.21.15.196:6836/1529420259,v1:172.21.15.196:6837/1529420259] conn(0x5642d3587cd0 0x5642d35869e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:35:57.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.392+0000 7f78bda93b80 4 client.14970 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T20:35:57.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.392+0000 7f78bda93b80 1 -- 172.21.15.181:0/945969808 --> [v2:172.21.15.196:6836/1529420259,v1:172.21.15.196:6837/1529420259] -- command(tid 0: {"prefix": "status"}) -- 0x5642d3454370 con 0x5642d3587cd0 2024-05-03T20:35:57.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.392+0000 7f78abfff700 1 --2- 172.21.15.181:0/945969808 >> [v2:172.21.15.196:6836/1529420259,v1:172.21.15.196:6837/1529420259] conn(0x5642d3587cd0 0x5642d35869e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:35:57.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.393+0000 7f788bfff700 10 client.14970 ms_handle_connect on v2:172.21.15.196:6836/1529420259 2024-05-03T20:35:57.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.393+0000 7f78abfff700 1 --2- 172.21.15.181:0/945969808 >> [v2:172.21.15.196:6836/1529420259,v1:172.21.15.196:6837/1529420259] conn(0x5642d3587cd0 0x5642d35869e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a4ea215ffac47c1a server_cookie=7d973505add92ece in_seq=0 out_seq=0 2024-05-03T20:35:57.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.393+0000 7f788bfff700 1 -- 172.21.15.181:0/945969808 <== mds.2 v2:172.21.15.196:6836/1529420259 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5642d3454370 con 0x5642d3587cd0 2024-05-03T20:35:57.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.393+0000 7f788bfff700 10 client.14970 handle_command_reply: tid=0 2024-05-03T20:35:57.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.393+0000 7f78bda93b80 1 client.14970 shutdown 2024-05-03T20:35:57.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.393+0000 7f78bda93b80 1 -- 172.21.15.181:0/945969808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d34fe8e0 msgr2=0x5642d34fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:57.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.393+0000 7f78bda93b80 1 --2- 172.21.15.181:0/945969808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d34fe8e0 0x5642d34fff90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f78a00a5890 tx=0x7f78a00096c0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.393+0000 7f78bda93b80 1 -- 172.21.15.181:0/945969808 shutdown_connections 2024-05-03T20:35:57.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.393+0000 7f78bda93b80 1 --2- 172.21.15.181:0/945969808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d3504620 0x5642d3500a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.393+0000 7f78bda93b80 1 --2- 172.21.15.181:0/945969808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5642d34fe8e0 0x5642d34fff90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.393+0000 7f78bda93b80 1 --2- 172.21.15.181:0/945969808 >> [v2:172.21.15.196:6836/1529420259,v1:172.21.15.196:6837/1529420259] conn(0x5642d3587cd0 0x5642d35869e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.393+0000 7f78bda93b80 1 --2- 172.21.15.181:0/945969808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5642d3503d00 0x5642d35004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.393+0000 7f78bda93b80 1 -- 172.21.15.181:0/945969808 >> 172.21.15.181:0/945969808 conn(0x5642d3426b20 msgr2=0x5642d3505f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:57.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.393+0000 7f788bfff700 0 client.14970 ms_handle_reset on v2:172.21.15.196:6836/1529420259 2024-05-03T20:35:57.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.393+0000 7f78bda93b80 1 -- 172.21.15.181:0/945969808 shutdown_connections 2024-05-03T20:35:57.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.393+0000 7f78bda93b80 1 -- 172.21.15.181:0/945969808 wait complete. 2024-05-03T20:35:57.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.393+0000 7f78bda93b80 20 client.14970 trim_cache size 0 max 16384 2024-05-03T20:35:57.401 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-05-03T20:35:57.401 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "1cd058ee-098b-11ef-bc97-c7b262605968", 2024-05-03T20:35:57.402 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 2, 2024-05-03T20:35:57.402 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 24464, 2024-05-03T20:35:57.402 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2024-05-03T20:35:57.402 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2024-05-03T20:35:57.402 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2024-05-03T20:35:57.402 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 121.935201095, 2024-05-03T20:35:57.403 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 27, 2024-05-03T20:35:57.403 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 83, 2024-05-03T20:35:57.403 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 81, 2024-05-03T20:35:57.403 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 161.73590992699999 2024-05-03T20:35:57.403 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-05-03T20:35:57.404 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.394+0000 7f78b3200700 1 -- 172.21.15.181:0/718017832 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f789c0b2910 msgr2=0x7f789c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:57.404 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.394+0000 7f78b3200700 1 --2- 172.21.15.181:0/718017832 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f789c0b2910 0x7f789c0b4dd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f78ac1030a0 tx=0x7f7898005cf0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.394+0000 7f78b3200700 1 -- 172.21.15.181:0/718017832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78ac103750 msgr2=0x7f78ac19f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:35:57.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.394+0000 7f78b3200700 1 --2- 172.21.15.181:0/718017832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78ac103750 0x7f78ac19f700 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f78a000d980 tx=0x7f78a000dd40 comp rx=0 tx=0).stop 2024-05-03T20:35:57.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.394+0000 7f78b3200700 1 -- 172.21.15.181:0/718017832 shutdown_connections 2024-05-03T20:35:57.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.394+0000 7f78b3200700 1 --2- 172.21.15.181:0/718017832 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f789c0b2910 0x7f789c0b4dd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.394+0000 7f78b3200700 1 --2- 172.21.15.181:0/718017832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78ac108850 0x7f78ac1998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.394+0000 7f78b3200700 1 --2- 172.21.15.181:0/718017832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78ac103750 0x7f78ac19f700 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.394+0000 7f78b3200700 1 --2- 172.21.15.181:0/718017832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78ac104100 0x7f78ac19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:35:57.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.394+0000 7f78b3200700 1 -- 172.21.15.181:0/718017832 >> 172.21.15.181:0/718017832 conn(0x7f78ac07a600 msgr2=0x7f78ac1017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:35:57.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.394+0000 7f78b3200700 1 -- 172.21.15.181:0/718017832 shutdown_connections 2024-05-03T20:35:57.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:35:57.394+0000 7f78b3200700 1 -- 172.21.15.181:0/718017832 wait complete. 2024-05-03T20:35:57.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:57 smithi181 ceph-mon[29801]: pgmap v354: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:35:57.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:57 smithi196 ceph-mon[32858]: pgmap v354: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:35:57.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:57 smithi204 ceph-mon[33595]: pgmap v354: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:35:58.113 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-03T20:35:58.367 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:35:59.754 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:59 smithi181 ceph-mon[29801]: pgmap v355: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:35:59.755 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:35:59 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:35:59.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:59 smithi196 ceph-mon[32858]: pgmap v355: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:35:59.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:35:59 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:35:59.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:59 smithi204 ceph-mon[33595]: pgmap v355: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:35:59.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:35:59 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:36:00.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.016+0000 7f09448d3700 1 -- 172.21.15.181:0/4020054503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0940108400 msgr2=0x7f09401087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:00.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.016+0000 7f09448d3700 1 --2- 172.21.15.181:0/4020054503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0940108400 0x7f09401087e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0930009990 tx=0x7f0930009ca0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.016+0000 7f09448d3700 1 -- 172.21.15.181:0/4020054503 shutdown_connections 2024-05-03T20:36:00.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.016+0000 7f09448d3700 1 --2- 172.21.15.181:0/4020054503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0940102d50 0x7f094010f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.016+0000 7f09448d3700 1 --2- 172.21.15.181:0/4020054503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0940108400 0x7f09401087e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.016+0000 7f09448d3700 1 --2- 172.21.15.181:0/4020054503 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0940102390 0x7f0940102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.016+0000 7f09448d3700 1 -- 172.21.15.181:0/4020054503 >> 172.21.15.181:0/4020054503 conn(0x7f0940079310 msgr2=0x7f0940079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:00.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.016+0000 7f09448d3700 1 -- 172.21.15.181:0/4020054503 shutdown_connections 2024-05-03T20:36:00.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.016+0000 7f09448d3700 1 -- 172.21.15.181:0/4020054503 wait complete. 2024-05-03T20:36:00.017 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.017+0000 7f09448d3700 1 Processor -- start 2024-05-03T20:36:00.017 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.017+0000 7f09448d3700 1 -- start start 2024-05-03T20:36:00.017 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.017+0000 7f09448d3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0940102390 0x7f09401987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:00.018 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.017+0000 7f09448d3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0940102d50 0x7f0940198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:00.018 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.017+0000 7f09448d3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0940108400 0x7f0940199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:00.018 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.017+0000 7f09448d3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0940112550 con 0x7f0940102d50 2024-05-03T20:36:00.018 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.017+0000 7f09448d3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f09401123d0 con 0x7f0940102390 2024-05-03T20:36:00.018 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.017+0000 7f09448d3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f09401126d0 con 0x7f0940108400 2024-05-03T20:36:00.018 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.017+0000 7f093dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0940102d50 0x7f0940198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:00.018 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.017+0000 7f093e59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0940102390 0x7f09401987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:00.018 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.017+0000 7f093dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0940102d50 0x7f0940198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58156/0 (socket says 172.21.15.181:58156) 2024-05-03T20:36:00.018 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.017+0000 7f093dd9b700 1 -- 172.21.15.181:0/1485354129 learned_addr learned my addr 172.21.15.181:0/1485354129 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:00.018 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.017+0000 7f093ed9d700 1 --2- 172.21.15.181:0/1485354129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0940108400 0x7f0940199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:00.018 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.018+0000 7f093dd9b700 1 -- 172.21.15.181:0/1485354129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0940108400 msgr2=0x7f0940199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:00.018 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.018+0000 7f093dd9b700 1 --2- 172.21.15.181:0/1485354129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0940108400 0x7f0940199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.018 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.018+0000 7f093dd9b700 1 -- 172.21.15.181:0/1485354129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0940102390 msgr2=0x7f09401987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:00.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.018+0000 7f093dd9b700 1 --2- 172.21.15.181:0/1485354129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0940102390 0x7f09401987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.018+0000 7f093dd9b700 1 -- 172.21.15.181:0/1485354129 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09401a8030 con 0x7f0940102d50 2024-05-03T20:36:00.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.018+0000 7f093e59c700 1 --2- 172.21.15.181:0/1485354129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0940102390 0x7f09401987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T20:36:00.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.018+0000 7f093ed9d700 1 --2- 172.21.15.181:0/1485354129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0940108400 0x7f0940199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T20:36:00.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.018+0000 7f093dd9b700 1 --2- 172.21.15.181:0/1485354129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0940102d50 0x7f0940198d00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f093400e9a0 tx=0x7f093400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:00.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.018+0000 7f092f7fe700 1 -- 172.21.15.181:0/1485354129 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093400ca10 con 0x7f0940102d50 2024-05-03T20:36:00.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.018+0000 7f092f7fe700 1 -- 172.21.15.181:0/1485354129 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f093400cbb0 con 0x7f0940102d50 2024-05-03T20:36:00.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.018+0000 7f09448d3700 1 -- 172.21.15.181:0/1485354129 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09401a8320 con 0x7f0940102d50 2024-05-03T20:36:00.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.018+0000 7f092f7fe700 1 -- 172.21.15.181:0/1485354129 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934018680 con 0x7f0940102d50 2024-05-03T20:36:00.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.018+0000 7f09448d3700 1 -- 172.21.15.181:0/1485354129 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09401a88b0 con 0x7f0940102d50 2024-05-03T20:36:00.020 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.021+0000 7f092f7fe700 1 -- 172.21.15.181:0/1485354129 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f0934010430 con 0x7f0940102d50 2024-05-03T20:36:00.020 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.021+0000 7f092f7fe700 1 --2- 172.21.15.181:0/1485354129 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f09280b2990 0x7f09280b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:00.020 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.021+0000 7f093e59c700 1 --2- 172.21.15.181:0/1485354129 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f09280b2990 0x7f09280b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:00.020 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.021+0000 7f092f7fe700 1 -- 172.21.15.181:0/1485354129 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f09340e2490 con 0x7f0940102d50 2024-05-03T20:36:00.020 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.022+0000 7f093e59c700 1 --2- 172.21.15.181:0/1485354129 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f09280b2990 0x7f09280b4e50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f09300098e0 tx=0x7f0930005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:00.022 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.022+0000 7f094f166b80 1 Processor -- start 2024-05-03T20:36:00.022 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.022+0000 7f094f166b80 1 -- start start 2024-05-03T20:36:00.022 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.023+0000 7f094f166b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9fee33c10 0x55a9fee33ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:00.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.023+0000 7f094f166b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fee34530 0x55a9fee37cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:00.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.023+0000 7f094f166b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fee38230 0x55a9fee38690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:00.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.023+0000 7f094f166b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a9fed86810 con 0x55a9fee33c10 2024-05-03T20:36:00.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.023+0000 7f094f166b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a9fed25770 con 0x55a9fee34530 2024-05-03T20:36:00.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.023+0000 7f094f166b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a9fed827d0 con 0x55a9fee38230 2024-05-03T20:36:00.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.023+0000 7f093e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9fee33c10 0x55a9fee33ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:00.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.023+0000 7f093dd9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fee34530 0x55a9fee37cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:00.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.023+0000 7f093dd9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fee34530 0x55a9fee37cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.181:42134/0 (socket says 172.21.15.181:42134) 2024-05-03T20:36:00.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.023+0000 7f093e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9fee33c10 0x55a9fee33ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58158/0 (socket says 172.21.15.181:58158) 2024-05-03T20:36:00.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.023+0000 7f093dd9b700 1 -- 172.21.15.181:0/500143322 learned_addr learned my addr 172.21.15.181:0/500143322 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:00.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.023+0000 7f093ed9d700 1 --2- 172.21.15.181:0/500143322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fee38230 0x55a9fee38690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:00.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.023+0000 7f093dd9b700 1 -- 172.21.15.181:0/500143322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fee38230 msgr2=0x55a9fee38690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:00.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.023+0000 7f093dd9b700 1 --2- 172.21.15.181:0/500143322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fee38230 0x55a9fee38690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.023+0000 7f093dd9b700 1 -- 172.21.15.181:0/500143322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9fee33c10 msgr2=0x55a9fee33ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:00.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.023+0000 7f093dd9b700 1 --2- 172.21.15.181:0/500143322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9fee33c10 0x55a9fee33ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.023+0000 7f093dd9b700 1 -- 172.21.15.181:0/500143322 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9fecd3880 con 0x55a9fee34530 2024-05-03T20:36:00.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.023+0000 7f093ed9d700 1 --2- 172.21.15.181:0/500143322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fee38230 0x55a9fee38690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:36:00.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.023+0000 7f093dd9b700 1 --2- 172.21.15.181:0/500143322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fee34530 0x55a9fee37cf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0934000c00 tx=0x7f09340b21c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d734fa92518cd316 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:00.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.024+0000 7f092cff9700 1 -- 172.21.15.181:0/500143322 <== mon.2 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09340aa050 con 0x55a9fee34530 2024-05-03T20:36:00.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.024+0000 7f092cff9700 1 -- 172.21.15.181:0/500143322 <== mon.2 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09340a9950 con 0x55a9fee34530 2024-05-03T20:36:00.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.024+0000 7f092cff9700 1 -- 172.21.15.181:0/500143322 <== mon.2 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09340b2320 con 0x55a9fee34530 2024-05-03T20:36:00.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.024+0000 7f094f166b80 1 -- 172.21.15.181:0/500143322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fee34530 msgr2=0x55a9fee37cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:00.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.024+0000 7f094f166b80 1 --2- 172.21.15.181:0/500143322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fee34530 0x55a9fee37cf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0934000c00 tx=0x7f09340b21c0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.024+0000 7f094f166b80 1 -- 172.21.15.181:0/500143322 shutdown_connections 2024-05-03T20:36:00.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.024+0000 7f094f166b80 1 --2- 172.21.15.181:0/500143322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fee38230 0x55a9fee38690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.024+0000 7f094f166b80 1 --2- 172.21.15.181:0/500143322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9fee33c10 0x55a9fee33ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.024+0000 7f094f166b80 1 --2- 172.21.15.181:0/500143322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fee34530 0x55a9fee37cf0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.024+0000 7f094f166b80 1 -- 172.21.15.181:0/500143322 >> 172.21.15.181:0/500143322 conn(0x55a9fed600b0 msgr2=0x55a9fee2e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:00.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.025+0000 7f094f166b80 1 -- 172.21.15.181:0/500143322 shutdown_connections 2024-05-03T20:36:00.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.025+0000 7f094f166b80 1 -- 172.21.15.181:0/500143322 wait complete. 2024-05-03T20:36:00.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.025+0000 7f094f166b80 1 -- start start 2024-05-03T20:36:00.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.025+0000 7f094f166b80 1 Processor -- start 2024-05-03T20:36:00.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.025+0000 7f094f166b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9fee33c10 0x55a9fee3f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:00.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.025+0000 7f094f166b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fee34530 0x55a9fee3ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:00.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.025+0000 7f094f166b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fee38230 0x55a9fee3cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:00.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.025+0000 7f094f166b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a9fee40490 con 0x55a9fee33c10 2024-05-03T20:36:00.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.025+0000 7f094f166b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a9fececbe0 con 0x55a9fee38230 2024-05-03T20:36:00.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.025+0000 7f094f166b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a9fee40610 con 0x55a9fee34530 2024-05-03T20:36:00.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.025+0000 7f094f166b80 10 client.0 _ll_register_callbacks cb 0x55a9fed1e8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T20:36:00.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.025+0000 7f093ed9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fee38230 0x55a9fee3cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:00.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.025+0000 7f093e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9fee33c10 0x55a9fee3f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:00.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.026+0000 7f093dd9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fee34530 0x55a9fee3ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:00.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.026+0000 7f093e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9fee33c10 0x55a9fee3f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58174/0 (socket says 172.21.15.181:58174) 2024-05-03T20:36:00.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.026+0000 7f093ed9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fee38230 0x55a9fee3cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.181:42142/0 (socket says 172.21.15.181:42142) 2024-05-03T20:36:00.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.026+0000 7f093e59c700 1 -- 172.21.15.181:0/3595461007 learned_addr learned my addr 172.21.15.181:0/3595461007 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:00.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.026+0000 7f093ed9d700 1 -- 172.21.15.181:0/3595461007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fee34530 msgr2=0x55a9fee3ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:00.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.026+0000 7f093ed9d700 1 --2- 172.21.15.181:0/3595461007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fee34530 0x55a9fee3ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.026+0000 7f093ed9d700 1 -- 172.21.15.181:0/3595461007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9fee33c10 msgr2=0x55a9fee3f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:00.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.026+0000 7f093ed9d700 1 --2- 172.21.15.181:0/3595461007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9fee33c10 0x55a9fee3f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.026+0000 7f093ed9d700 1 -- 172.21.15.181:0/3595461007 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9fed22640 con 0x55a9fee38230 2024-05-03T20:36:00.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.026+0000 7f093dd9b700 1 --2- 172.21.15.181:0/3595461007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fee34530 0x55a9fee3ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T20:36:00.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.026+0000 7f093e59c700 1 --2- 172.21.15.181:0/3595461007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9fee33c10 0x55a9fee3f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T20:36:00.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.026+0000 7f093ed9d700 1 --2- 172.21.15.181:0/3595461007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fee38230 0x55a9fee3cfd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f09380042f0 tx=0x7f0938004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7bf9f80ac06ee8e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:00.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.026+0000 7f0925ffb700 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-05-03T20:36:00.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.026+0000 7f0925ffb700 1 -- 172.21.15.181:0/3595461007 <== mon.2 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093801a030 con 0x55a9fee38230 2024-05-03T20:36:00.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.026+0000 7f0925ffb700 1 -- 172.21.15.181:0/3595461007 <== mon.2 v2:172.21.15.196:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f0938004bb0 con 0x55a9fee38230 2024-05-03T20:36:00.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.026+0000 7f094f166b80 1 -- 172.21.15.181:0/3595461007 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09340a9950 con 0x55a9fee38230 2024-05-03T20:36:00.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.026+0000 7f0925ffb700 1 -- 172.21.15.181:0/3595461007 <== mon.2 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0938026050 con 0x55a9fee38230 2024-05-03T20:36:00.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.026+0000 7f094f166b80 10 client.24695 fetch_fsmap learned FSMap version 27 2024-05-03T20:36:00.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.026+0000 7f094f166b80 1 -- 172.21.15.181:0/3595461007 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a9fee4ad30 con 0x55a9fee38230 2024-05-03T20:36:00.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.027+0000 7f0925ffb700 1 -- 172.21.15.181:0/3595461007 <== mon.2 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09340a9950 con 0x55a9fee38230 2024-05-03T20:36:00.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.027+0000 7f0925ffb700 1 -- 172.21.15.181:0/3595461007 <== mon.2 v2:172.21.15.196:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f093800e760 con 0x55a9fee38230 2024-05-03T20:36:00.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.027+0000 7f094f166b80 10 client.24695 fetch_fsmap finished waiting for FSMap version 27 2024-05-03T20:36:00.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.027+0000 7f094f166b80 10 client.24695 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-03T20:36:00.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.027+0000 7f094f166b80 20 client.24695 populate_metadata read hostname 'smithi181' 2024-05-03T20:36:00.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.027+0000 7f094f166b80 1 --2- 172.21.15.181:0/3595461007 >> [v2:172.21.15.204:6838/1644828559,v1:172.21.15.204:6839/1644828559] conn(0x55a9fee4d340 0x55a9fee4e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:00.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.027+0000 7f094f166b80 4 client.24695 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T20:36:00.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.027+0000 7f094f166b80 1 -- 172.21.15.181:0/3595461007 --> [v2:172.21.15.204:6838/1644828559,v1:172.21.15.204:6839/1644828559] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9fed941f0 con 0x55a9fee4d340 2024-05-03T20:36:00.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.027+0000 7f093e59c700 1 --2- 172.21.15.181:0/3595461007 >> [v2:172.21.15.204:6838/1644828559,v1:172.21.15.204:6839/1644828559] conn(0x55a9fee4d340 0x55a9fee4e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:00.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.027+0000 7f0925ffb700 10 client.24695 ms_handle_connect on v2:172.21.15.204:6838/1644828559 2024-05-03T20:36:00.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.027+0000 7f093e59c700 1 --2- 172.21.15.181:0/3595461007 >> [v2:172.21.15.204:6838/1644828559,v1:172.21.15.204:6839/1644828559] conn(0x55a9fee4d340 0x55a9fee4e7b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fa8ab75067eb01e6 server_cookie=d9921f4caa47f6c in_seq=0 out_seq=0 2024-05-03T20:36:00.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.028+0000 7f0925ffb700 1 -- 172.21.15.181:0/3595461007 <== mds.1 v2:172.21.15.204:6838/1644828559 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a9fed941f0 con 0x55a9fee4d340 2024-05-03T20:36:00.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.028+0000 7f0925ffb700 10 client.24695 handle_command_reply: tid=0 2024-05-03T20:36:00.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.028+0000 7f094f166b80 1 client.24695 shutdown 2024-05-03T20:36:00.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.028+0000 7f094f166b80 1 -- 172.21.15.181:0/3595461007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fee38230 msgr2=0x55a9fee3cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:00.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.028+0000 7f094f166b80 1 --2- 172.21.15.181:0/3595461007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fee38230 0x55a9fee3cfd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f09380042f0 tx=0x7f0938004320 comp rx=0 tx=0).stop 2024-05-03T20:36:00.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.028+0000 7f094f166b80 1 -- 172.21.15.181:0/3595461007 shutdown_connections 2024-05-03T20:36:00.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.028+0000 7f094f166b80 1 --2- 172.21.15.181:0/3595461007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fee38230 0x55a9fee3cfd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.028+0000 7f094f166b80 1 --2- 172.21.15.181:0/3595461007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9fee33c10 0x55a9fee3f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.028+0000 7f094f166b80 1 --2- 172.21.15.181:0/3595461007 >> [v2:172.21.15.204:6838/1644828559,v1:172.21.15.204:6839/1644828559] conn(0x55a9fee4d340 0x55a9fee4e7b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.028+0000 7f094f166b80 1 --2- 172.21.15.181:0/3595461007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fee34530 0x55a9fee3ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.028+0000 7f0925ffb700 0 client.24695 ms_handle_reset on v2:172.21.15.204:6838/1644828559 2024-05-03T20:36:00.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.028+0000 7f094f166b80 1 -- 172.21.15.181:0/3595461007 >> 172.21.15.181:0/3595461007 conn(0x55a9fed600b0 msgr2=0x55a9fee396e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:00.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.029+0000 7f094f166b80 1 -- 172.21.15.181:0/3595461007 shutdown_connections 2024-05-03T20:36:00.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.029+0000 7f094f166b80 1 -- 172.21.15.181:0/3595461007 wait complete. 2024-05-03T20:36:00.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.029+0000 7f094f166b80 20 client.24695 trim_cache size 0 max 16384 2024-05-03T20:36:00.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.050+0000 7f094f166b80 1 Processor -- start 2024-05-03T20:36:00.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.051+0000 7f094f166b80 1 -- start start 2024-05-03T20:36:00.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.051+0000 7f094f166b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fee33c10 0x55a9fee3a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:00.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.051+0000 7f094f166b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9fee34530 0x55a9fee38de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:00.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.051+0000 7f094f166b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fee38230 0x55a9fee39320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:00.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.051+0000 7f094f166b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a9fee4caf0 con 0x55a9fee34530 2024-05-03T20:36:00.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.051+0000 7f094f166b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a9fee4d1c0 con 0x55a9fee33c10 2024-05-03T20:36:00.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.051+0000 7f094f166b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a9fee4d890 con 0x55a9fee38230 2024-05-03T20:36:00.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.051+0000 7f093dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9fee34530 0x55a9fee38de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:00.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.051+0000 7f093dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9fee34530 0x55a9fee38de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58190/0 (socket says 172.21.15.181:58190) 2024-05-03T20:36:00.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.051+0000 7f093dd9b700 1 -- 172.21.15.181:0/643888155 learned_addr learned my addr 172.21.15.181:0/643888155 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:00.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.051+0000 7f093e59c700 1 --2- 172.21.15.181:0/643888155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fee33c10 0x55a9fee3a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:00.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.051+0000 7f093ed9d700 1 --2- 172.21.15.181:0/643888155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fee38230 0x55a9fee39320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:00.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.051+0000 7f093dd9b700 1 -- 172.21.15.181:0/643888155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fee38230 msgr2=0x55a9fee39320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:00.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.051+0000 7f093dd9b700 1 --2- 172.21.15.181:0/643888155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fee38230 0x55a9fee39320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.051+0000 7f093dd9b700 1 -- 172.21.15.181:0/643888155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fee33c10 msgr2=0x55a9fee3a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:00.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.051+0000 7f093dd9b700 1 --2- 172.21.15.181:0/643888155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fee33c10 0x55a9fee3a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.051+0000 7f093dd9b700 1 -- 172.21.15.181:0/643888155 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9fee2ea10 con 0x55a9fee34530 2024-05-03T20:36:00.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.051+0000 7f093dd9b700 1 --2- 172.21.15.181:0/643888155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9fee34530 0x55a9fee38de0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f093400e970 tx=0x7f0934010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21de5c8e13a41116 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:00.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.052+0000 7f0927fff700 1 -- 172.21.15.181:0/643888155 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09340c3070 con 0x55a9fee34530 2024-05-03T20:36:00.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.052+0000 7f0927fff700 1 -- 172.21.15.181:0/643888155 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09340aa050 con 0x55a9fee34530 2024-05-03T20:36:00.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.052+0000 7f0927fff700 1 -- 172.21.15.181:0/643888155 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09340b2320 con 0x55a9fee34530 2024-05-03T20:36:00.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.052+0000 7f094f166b80 1 -- 172.21.15.181:0/643888155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9fee34530 msgr2=0x55a9fee38de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:00.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.052+0000 7f094f166b80 1 --2- 172.21.15.181:0/643888155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9fee34530 0x55a9fee38de0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f093400e970 tx=0x7f0934010c10 comp rx=0 tx=0).stop 2024-05-03T20:36:00.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.052+0000 7f094f166b80 1 -- 172.21.15.181:0/643888155 shutdown_connections 2024-05-03T20:36:00.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.052+0000 7f094f166b80 1 --2- 172.21.15.181:0/643888155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fee38230 0x55a9fee39320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.052+0000 7f094f166b80 1 --2- 172.21.15.181:0/643888155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fee33c10 0x55a9fee3a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.052+0000 7f094f166b80 1 --2- 172.21.15.181:0/643888155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9fee34530 0x55a9fee38de0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.052+0000 7f094f166b80 1 -- 172.21.15.181:0/643888155 >> 172.21.15.181:0/643888155 conn(0x55a9fed600b0 msgr2=0x55a9fee39ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:00.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.052+0000 7f094f166b80 1 -- 172.21.15.181:0/643888155 shutdown_connections 2024-05-03T20:36:00.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.052+0000 7f094f166b80 1 -- 172.21.15.181:0/643888155 wait complete. 2024-05-03T20:36:00.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.053+0000 7f094f166b80 1 -- start start 2024-05-03T20:36:00.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.053+0000 7f094f166b80 1 Processor -- start 2024-05-03T20:36:00.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.053+0000 7f094f166b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9fee33c10 0x55a9fee36310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:00.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.053+0000 7f094f166b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fee34530 0x55a9fee57c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:00.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.053+0000 7f094f166b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fee38230 0x55a9fee58160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:00.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.053+0000 7f094f166b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a9fee4e630 con 0x55a9fee33c10 2024-05-03T20:36:00.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.053+0000 7f094f166b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a9fee4df60 con 0x55a9fee38230 2024-05-03T20:36:00.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.053+0000 7f094f166b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a9fee4ed00 con 0x55a9fee34530 2024-05-03T20:36:00.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.053+0000 7f094f166b80 10 client.0 _ll_register_callbacks cb 0x55a9fed1e8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T20:36:00.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.053+0000 7f093dd9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fee34530 0x55a9fee57c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:00.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.053+0000 7f093e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9fee33c10 0x55a9fee36310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:00.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.053+0000 7f093ed9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fee38230 0x55a9fee58160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:00.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.053+0000 7f093dd9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fee34530 0x55a9fee57c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.181:51716/0 (socket says 172.21.15.181:51716) 2024-05-03T20:36:00.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.053+0000 7f093dd9b700 1 -- 172.21.15.181:0/607112019 learned_addr learned my addr 172.21.15.181:0/607112019 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:00.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.053+0000 7f093ed9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fee38230 0x55a9fee58160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.181:42158/0 (socket says 172.21.15.181:42158) 2024-05-03T20:36:00.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.053+0000 7f093dd9b700 1 -- 172.21.15.181:0/607112019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fee38230 msgr2=0x55a9fee58160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:00.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.053+0000 7f093dd9b700 1 --2- 172.21.15.181:0/607112019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fee38230 0x55a9fee58160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.053+0000 7f093dd9b700 1 -- 172.21.15.181:0/607112019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9fee33c10 msgr2=0x55a9fee36310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:00.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.053+0000 7f093dd9b700 1 --2- 172.21.15.181:0/607112019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9fee33c10 0x55a9fee36310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.053+0000 7f093dd9b700 1 -- 172.21.15.181:0/607112019 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9fee34e10 con 0x55a9fee34530 2024-05-03T20:36:00.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.054+0000 7f093dd9b700 1 --2- 172.21.15.181:0/607112019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fee34530 0x55a9fee57c20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f09340a59d0 tx=0x7f09340a9850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e74e8a400d7675e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:00.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.054+0000 7f0925ffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-05-03T20:36:00.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.054+0000 7f0925ffb700 1 -- 172.21.15.181:0/607112019 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09340c3070 con 0x55a9fee34530 2024-05-03T20:36:00.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.054+0000 7f0925ffb700 1 -- 172.21.15.181:0/607112019 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f09340b2530 con 0x55a9fee34530 2024-05-03T20:36:00.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.054+0000 7f094f166b80 1 -- 172.21.15.181:0/607112019 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09340aa050 con 0x55a9fee34530 2024-05-03T20:36:00.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.054+0000 7f0925ffb700 1 -- 172.21.15.181:0/607112019 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0934089050 con 0x55a9fee34530 2024-05-03T20:36:00.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.054+0000 7f094f166b80 10 client.24718 fetch_fsmap learned FSMap version 27 2024-05-03T20:36:00.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.054+0000 7f094f166b80 1 -- 172.21.15.181:0/607112019 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a9feeb8a30 con 0x55a9fee34530 2024-05-03T20:36:00.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.054+0000 7f0925ffb700 1 -- 172.21.15.181:0/607112019 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09340b27b0 con 0x55a9fee34530 2024-05-03T20:36:00.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.054+0000 7f0925ffb700 1 -- 172.21.15.181:0/607112019 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f09340a8620 con 0x55a9fee34530 2024-05-03T20:36:00.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.054+0000 7f094f166b80 10 client.24718 fetch_fsmap finished waiting for FSMap version 27 2024-05-03T20:36:00.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.054+0000 7f094f166b80 10 client.24718 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-03T20:36:00.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.054+0000 7f094f166b80 20 client.24718 populate_metadata read hostname 'smithi181' 2024-05-03T20:36:00.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.054+0000 7f094f166b80 1 --2- 172.21.15.181:0/607112019 >> [v2:172.21.15.204:6838/1644828559,v1:172.21.15.204:6839/1644828559] conn(0x55a9feebccb0 0x55a9feebb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:00.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.054+0000 7f094f166b80 4 client.24718 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T20:36:00.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.054+0000 7f094f166b80 1 -- 172.21.15.181:0/607112019 --> [v2:172.21.15.204:6838/1644828559,v1:172.21.15.204:6839/1644828559] -- command(tid 0: {"prefix": "status"}) -- 0x7f09340aa050 con 0x55a9feebccb0 2024-05-03T20:36:00.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.055+0000 7f093e59c700 1 --2- 172.21.15.181:0/607112019 >> [v2:172.21.15.204:6838/1644828559,v1:172.21.15.204:6839/1644828559] conn(0x55a9feebccb0 0x55a9feebb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:00.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.055+0000 7f093e59c700 1 --2- 172.21.15.181:0/607112019 >> [v2:172.21.15.204:6838/1644828559,v1:172.21.15.204:6839/1644828559] conn(0x55a9feebccb0 0x55a9feebb9c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1ec9f88a409118c5 server_cookie=5c45df506d7a2404 in_seq=0 out_seq=0 2024-05-03T20:36:00.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.055+0000 7f0925ffb700 10 client.24718 ms_handle_connect on v2:172.21.15.204:6838/1644828559 2024-05-03T20:36:00.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.055+0000 7f0925ffb700 1 -- 172.21.15.181:0/607112019 <== mds.1 v2:172.21.15.204:6838/1644828559 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x7f09340aa050 con 0x55a9feebccb0 2024-05-03T20:36:00.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.055+0000 7f0925ffb700 10 client.24718 handle_command_reply: tid=0 2024-05-03T20:36:00.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.055+0000 7f094f166b80 1 client.24718 shutdown 2024-05-03T20:36:00.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.055+0000 7f094f166b80 1 -- 172.21.15.181:0/607112019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fee34530 msgr2=0x55a9fee57c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:00.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.055+0000 7f094f166b80 1 --2- 172.21.15.181:0/607112019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fee34530 0x55a9fee57c20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f09340a59d0 tx=0x7f09340a9850 comp rx=0 tx=0).stop 2024-05-03T20:36:00.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.055+0000 7f094f166b80 1 -- 172.21.15.181:0/607112019 shutdown_connections 2024-05-03T20:36:00.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.055+0000 7f094f166b80 1 --2- 172.21.15.181:0/607112019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fee38230 0x55a9fee58160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.055+0000 7f094f166b80 1 --2- 172.21.15.181:0/607112019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a9fee33c10 0x55a9fee36310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.055+0000 7f094f166b80 1 --2- 172.21.15.181:0/607112019 >> [v2:172.21.15.204:6838/1644828559,v1:172.21.15.204:6839/1644828559] conn(0x55a9feebccb0 0x55a9feebb9c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.055+0000 7f094f166b80 1 --2- 172.21.15.181:0/607112019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fee34530 0x55a9fee57c20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.055+0000 7f0925ffb700 0 client.24718 ms_handle_reset on v2:172.21.15.204:6838/1644828559 2024-05-03T20:36:00.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.055+0000 7f094f166b80 1 -- 172.21.15.181:0/607112019 >> 172.21.15.181:0/607112019 conn(0x55a9fed600b0 msgr2=0x55a9fee32a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:00.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.055+0000 7f094f166b80 1 -- 172.21.15.181:0/607112019 shutdown_connections 2024-05-03T20:36:00.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.055+0000 7f094f166b80 1 -- 172.21.15.181:0/607112019 wait complete. 2024-05-03T20:36:00.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.056+0000 7f094f166b80 20 client.24718 trim_cache size 0 max 16384 2024-05-03T20:36:00.059 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-05-03T20:36:00.059 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "1cd058ee-098b-11ef-bc97-c7b262605968", 2024-05-03T20:36:00.059 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 1, 2024-05-03T20:36:00.059 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 24475, 2024-05-03T20:36:00.060 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2024-05-03T20:36:00.060 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2024-05-03T20:36:00.060 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2024-05-03T20:36:00.060 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 125.603562517, 2024-05-03T20:36:00.060 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 27, 2024-05-03T20:36:00.060 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 83, 2024-05-03T20:36:00.060 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 81, 2024-05-03T20:36:00.060 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 161.71720420400001 2024-05-03T20:36:00.060 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-05-03T20:36:00.060 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.056+0000 7f09448d3700 1 -- 172.21.15.181:0/1485354129 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f09280b2990 msgr2=0x7f09280b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:00.061 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.056+0000 7f09448d3700 1 --2- 172.21.15.181:0/1485354129 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f09280b2990 0x7f09280b4e50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f09300098e0 tx=0x7f0930005960 comp rx=0 tx=0).stop 2024-05-03T20:36:00.061 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.056+0000 7f09448d3700 1 -- 172.21.15.181:0/1485354129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0940102d50 msgr2=0x7f0940198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:00.061 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.056+0000 7f09448d3700 1 --2- 172.21.15.181:0/1485354129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0940102d50 0x7f0940198d00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f093400e9a0 tx=0x7f093400ecb0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.061 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.056+0000 7f09448d3700 1 -- 172.21.15.181:0/1485354129 shutdown_connections 2024-05-03T20:36:00.061 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.056+0000 7f09448d3700 1 --2- 172.21.15.181:0/1485354129 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f09280b2990 0x7f09280b4e50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.061 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.056+0000 7f09448d3700 1 --2- 172.21.15.181:0/1485354129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0940108400 0x7f0940199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.061 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.056+0000 7f09448d3700 1 --2- 172.21.15.181:0/1485354129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0940102390 0x7f09401987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.061 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.056+0000 7f09448d3700 1 --2- 172.21.15.181:0/1485354129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0940102d50 0x7f0940198d00 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:00.061 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.056+0000 7f09448d3700 1 -- 172.21.15.181:0/1485354129 >> 172.21.15.181:0/1485354129 conn(0x7f0940079310 msgr2=0x7f0940101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:00.061 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.056+0000 7f09448d3700 1 -- 172.21.15.181:0/1485354129 shutdown_connections 2024-05-03T20:36:00.061 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:00.056+0000 7f09448d3700 1 -- 172.21.15.181:0/1485354129 wait complete. 2024-05-03T20:36:00.776 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-03T20:36:01.523 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:36:01.839 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:01 smithi181 ceph-mon[29801]: pgmap v356: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:36:01.839 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:01 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:36:01.839 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:01 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:36:01.839 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:01 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:36:01.839 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:01 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:36:01.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:01 smithi196 ceph-mon[32858]: pgmap v356: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:36:01.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:01 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:36:01.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:01 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:36:01.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:01 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:36:01.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:01 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:36:01.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:01 smithi204 ceph-mon[33595]: pgmap v356: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:36:01.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:01 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:36:01.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:01 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:36:01.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:01 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:36:01.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:01 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:36:03.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.081+0000 7f3d4d603700 1 -- 172.21.15.181:0/1887929043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d48103eb0 msgr2=0x7f3d48110390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:03.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.081+0000 7f3d4d603700 1 --2- 172.21.15.181:0/1887929043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d48103eb0 0x7f3d48110390 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3d4000b0a0 tx=0x7f3d4000b3b0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.081+0000 7f3d4d603700 1 -- 172.21.15.181:0/1887929043 shutdown_connections 2024-05-03T20:36:03.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.081+0000 7f3d4d603700 1 --2- 172.21.15.181:0/1887929043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d48103eb0 0x7f3d48110390 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.081+0000 7f3d4d603700 1 --2- 172.21.15.181:0/1887929043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d48109600 0x7f3d481099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.081+0000 7f3d4d603700 1 --2- 172.21.15.181:0/1887929043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d481034f0 0x7f3d48103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.081+0000 7f3d4d603700 1 -- 172.21.15.181:0/1887929043 >> 172.21.15.181:0/1887929043 conn(0x7f3d4807a650 msgr2=0x7f3d4807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:03.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.081+0000 7f3d4d603700 1 -- 172.21.15.181:0/1887929043 shutdown_connections 2024-05-03T20:36:03.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.081+0000 7f3d4d603700 1 -- 172.21.15.181:0/1887929043 wait complete. 2024-05-03T20:36:03.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.082+0000 7f3d4d603700 1 Processor -- start 2024-05-03T20:36:03.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.082+0000 7f3d4d603700 1 -- start start 2024-05-03T20:36:03.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.082+0000 7f3d4d603700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d481034f0 0x7f3d480757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:03.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.082+0000 7f3d4d603700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d48103eb0 0x7f3d48075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:03.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.082+0000 7f3d4d603700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d48109600 0x7f3d48076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:03.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.082+0000 7f3d4d603700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3d481136d0 con 0x7f3d48103eb0 2024-05-03T20:36:03.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.082+0000 7f3d4d603700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3d48113550 con 0x7f3d48109600 2024-05-03T20:36:03.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.082+0000 7f3d4d603700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3d48113850 con 0x7f3d481034f0 2024-05-03T20:36:03.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.082+0000 7f3d467fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d48103eb0 0x7f3d48075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:03.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.083+0000 7f3d467fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d48103eb0 0x7f3d48075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58226/0 (socket says 172.21.15.181:58226) 2024-05-03T20:36:03.100 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.083+0000 7f3d467fc700 1 -- 172.21.15.181:0/3329801765 learned_addr learned my addr 172.21.15.181:0/3329801765 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:03.100 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.083+0000 7f3d477fe700 1 --2- 172.21.15.181:0/3329801765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d48109600 0x7f3d48076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:03.100 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.083+0000 7f3d46ffd700 1 --2- 172.21.15.181:0/3329801765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d481034f0 0x7f3d480757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:03.100 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.083+0000 7f3d467fc700 1 -- 172.21.15.181:0/3329801765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d481034f0 msgr2=0x7f3d480757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:03.100 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.083+0000 7f3d467fc700 1 --2- 172.21.15.181:0/3329801765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d481034f0 0x7f3d480757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.100 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.083+0000 7f3d467fc700 1 -- 172.21.15.181:0/3329801765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d48109600 msgr2=0x7f3d48076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:03.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.083+0000 7f3d467fc700 1 --2- 172.21.15.181:0/3329801765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d48109600 0x7f3d48076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.083+0000 7f3d467fc700 1 -- 172.21.15.181:0/3329801765 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d481a7ff0 con 0x7f3d48103eb0 2024-05-03T20:36:03.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.083+0000 7f3d467fc700 1 --2- 172.21.15.181:0/3329801765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d48103eb0 0x7f3d48075ce0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00d980 tx=0x7f3d3c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:03.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.083+0000 7f3d2ffff700 1 -- 172.21.15.181:0/3329801765 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c016070 con 0x7f3d48103eb0 2024-05-03T20:36:03.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.083+0000 7f3d4d603700 1 -- 172.21.15.181:0/3329801765 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d481a82e0 con 0x7f3d48103eb0 2024-05-03T20:36:03.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.083+0000 7f3d2ffff700 1 -- 172.21.15.181:0/3329801765 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d3c004830 con 0x7f3d48103eb0 2024-05-03T20:36:03.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.083+0000 7f3d2ffff700 1 -- 172.21.15.181:0/3329801765 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c005540 con 0x7f3d48103eb0 2024-05-03T20:36:03.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.083+0000 7f3d4d603700 1 -- 172.21.15.181:0/3329801765 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d481a8870 con 0x7f3d48103eb0 2024-05-03T20:36:03.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.085+0000 7f3d2ffff700 1 -- 172.21.15.181:0/3329801765 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f3d3c004220 con 0x7f3d48103eb0 2024-05-03T20:36:03.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.086+0000 7f3d2ffff700 1 --2- 172.21.15.181:0/3329801765 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3d300b2910 0x7f3d300b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:03.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.086+0000 7f3d46ffd700 1 --2- 172.21.15.181:0/3329801765 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3d300b2910 0x7f3d300b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:03.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.086+0000 7f3d2ffff700 1 -- 172.21.15.181:0/3329801765 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f3d3c0e2160 con 0x7f3d48103eb0 2024-05-03T20:36:03.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.087+0000 7f3d46ffd700 1 --2- 172.21.15.181:0/3329801765 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3d300b2910 0x7f3d300b4dd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3d38009a10 tx=0x7f3d38009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:03.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.087+0000 7f3d57e96b80 1 Processor -- start 2024-05-03T20:36:03.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.087+0000 7f3d57e96b80 1 -- start start 2024-05-03T20:36:03.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.087+0000 7f3d57e96b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f2d8f740 0x5615f2d8fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:03.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.087+0000 7f3d57e96b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615f2d90060 0x5615f2d8af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:03.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.087+0000 7f3d57e96b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8b480 0x5615f2d8b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:03.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.087+0000 7f3d57e96b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5615f2cdd8f0 con 0x5615f2d8b480 2024-05-03T20:36:03.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.087+0000 7f3d57e96b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5615f2c42d50 con 0x5615f2d8f740 2024-05-03T20:36:03.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.087+0000 7f3d57e96b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5615f2cd98b0 con 0x5615f2d90060 2024-05-03T20:36:03.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.087+0000 7f3d477fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8b480 0x5615f2d8b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:03.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.087+0000 7f3d46ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f2d8f740 0x5615f2d8fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:03.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.087+0000 7f3d477fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8b480 0x5615f2d8b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58236/0 (socket says 172.21.15.181:58236) 2024-05-03T20:36:03.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.087+0000 7f3d477fe700 1 -- 172.21.15.181:0/2577141615 learned_addr learned my addr 172.21.15.181:0/2577141615 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:03.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.087+0000 7f3d467fc700 1 --2- 172.21.15.181:0/2577141615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615f2d90060 0x5615f2d8af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:03.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.087+0000 7f3d477fe700 1 -- 172.21.15.181:0/2577141615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615f2d90060 msgr2=0x5615f2d8af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:03.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.087+0000 7f3d477fe700 1 --2- 172.21.15.181:0/2577141615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615f2d90060 0x5615f2d8af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.087+0000 7f3d477fe700 1 -- 172.21.15.181:0/2577141615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f2d8f740 msgr2=0x5615f2d8fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:03.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.087+0000 7f3d477fe700 1 --2- 172.21.15.181:0/2577141615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f2d8f740 0x5615f2d8fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.087+0000 7f3d477fe700 1 -- 172.21.15.181:0/2577141615 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615f2c2a880 con 0x5615f2d8b480 2024-05-03T20:36:03.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.087+0000 7f3d46ffd700 1 --2- 172.21.15.181:0/2577141615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f2d8f740 0x5615f2d8fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T20:36:03.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.087+0000 7f3d467fc700 1 --2- 172.21.15.181:0/2577141615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615f2d90060 0x5615f2d8af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:36:03.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.088+0000 7f3d477fe700 1 --2- 172.21.15.181:0/2577141615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8b480 0x5615f2d8b8c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3d40009310 tx=0x7f3d40009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6418d955a6987eb server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:03.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.088+0000 7f3d2d7fa700 1 -- 172.21.15.181:0/2577141615 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d40020070 con 0x5615f2d8b480 2024-05-03T20:36:03.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.088+0000 7f3d2d7fa700 1 -- 172.21.15.181:0/2577141615 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d40003c00 con 0x5615f2d8b480 2024-05-03T20:36:03.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.088+0000 7f3d2d7fa700 1 -- 172.21.15.181:0/2577141615 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d400052a0 con 0x5615f2d8b480 2024-05-03T20:36:03.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.089+0000 7f3d57e96b80 1 -- 172.21.15.181:0/2577141615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8b480 msgr2=0x5615f2d8b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:03.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.089+0000 7f3d57e96b80 1 --2- 172.21.15.181:0/2577141615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8b480 0x5615f2d8b8c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3d40009310 tx=0x7f3d40009340 comp rx=0 tx=0).stop 2024-05-03T20:36:03.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.089+0000 7f3d57e96b80 1 -- 172.21.15.181:0/2577141615 shutdown_connections 2024-05-03T20:36:03.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.089+0000 7f3d57e96b80 1 --2- 172.21.15.181:0/2577141615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8b480 0x5615f2d8b8c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.089+0000 7f3d57e96b80 1 --2- 172.21.15.181:0/2577141615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f2d8f740 0x5615f2d8fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.089+0000 7f3d57e96b80 1 --2- 172.21.15.181:0/2577141615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615f2d90060 0x5615f2d8af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.089+0000 7f3d57e96b80 1 -- 172.21.15.181:0/2577141615 >> 172.21.15.181:0/2577141615 conn(0x5615f2cb7b20 msgr2=0x5615f2d859a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:03.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.089+0000 7f3d57e96b80 1 -- 172.21.15.181:0/2577141615 shutdown_connections 2024-05-03T20:36:03.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.089+0000 7f3d57e96b80 1 -- 172.21.15.181:0/2577141615 wait complete. 2024-05-03T20:36:03.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.089+0000 7f3d57e96b80 1 -- start start 2024-05-03T20:36:03.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.089+0000 7f3d57e96b80 1 Processor -- start 2024-05-03T20:36:03.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.089+0000 7f3d57e96b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8b480 0x5615f2d96d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:03.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.089+0000 7f3d57e96b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f2d8f740 0x5615f2d94110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:03.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.089+0000 7f3d57e96b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615f2d90060 0x5615f2d94650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:03.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.089+0000 7f3d57e96b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5615f2d97b40 con 0x5615f2d8b480 2024-05-03T20:36:03.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.089+0000 7f3d57e96b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5615f2c96e80 con 0x5615f2d8f740 2024-05-03T20:36:03.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.089+0000 7f3d57e96b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5615f2d97cc0 con 0x5615f2d90060 2024-05-03T20:36:03.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.089+0000 7f3d57e96b80 10 client.0 _ll_register_callbacks cb 0x5615f2c71560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T20:36:03.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.089+0000 7f3d46ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8b480 0x5615f2d96d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:03.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.089+0000 7f3d46ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8b480 0x5615f2d96d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58240/0 (socket says 172.21.15.181:58240) 2024-05-03T20:36:03.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.089+0000 7f3d46ffd700 1 -- 172.21.15.181:0/2217587690 learned_addr learned my addr 172.21.15.181:0/2217587690 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:03.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.090+0000 7f3d467fc700 1 --2- 172.21.15.181:0/2217587690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f2d8f740 0x5615f2d94110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:03.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.090+0000 7f3d46ffd700 1 -- 172.21.15.181:0/2217587690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615f2d90060 msgr2=0x5615f2d94650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:03.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.090+0000 7f3d477fe700 1 --2- 172.21.15.181:0/2217587690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615f2d90060 0x5615f2d94650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:03.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.090+0000 7f3d46ffd700 1 --2- 172.21.15.181:0/2217587690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615f2d90060 0x5615f2d94650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.111 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.090+0000 7f3d46ffd700 1 -- 172.21.15.181:0/2217587690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f2d8f740 msgr2=0x5615f2d94110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:03.111 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.090+0000 7f3d46ffd700 1 --2- 172.21.15.181:0/2217587690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f2d8f740 0x5615f2d94110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.111 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.090+0000 7f3d46ffd700 1 -- 172.21.15.181:0/2217587690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615f2c94710 con 0x5615f2d8b480 2024-05-03T20:36:03.111 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.090+0000 7f3d467fc700 1 --2- 172.21.15.181:0/2217587690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f2d8f740 0x5615f2d94110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:36:03.111 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.090+0000 7f3d46ffd700 1 --2- 172.21.15.181:0/2217587690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8b480 0x5615f2d96d20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3d380154f0 tx=0x7f3d38004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6191c80acd82dedc server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:03.111 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.090+0000 7f3d2aff5700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-03T20:36:03.112 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.090+0000 7f3d2aff5700 1 -- 172.21.15.181:0/2217587690 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38025070 con 0x5615f2d8b480 2024-05-03T20:36:03.112 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.090+0000 7f3d2aff5700 1 -- 172.21.15.181:0/2217587690 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f3d38015d20 con 0x5615f2d8b480 2024-05-03T20:36:03.112 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.090+0000 7f3d57e96b80 1 -- 172.21.15.181:0/2217587690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d40003c00 con 0x5615f2d8b480 2024-05-03T20:36:03.112 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.090+0000 7f3d2aff5700 1 -- 172.21.15.181:0/2217587690 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d3801b040 con 0x5615f2d8b480 2024-05-03T20:36:03.112 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.090+0000 7f3d57e96b80 10 client.15000 fetch_fsmap learned FSMap version 27 2024-05-03T20:36:03.112 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.090+0000 7f3d57e96b80 1 -- 172.21.15.181:0/2217587690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5615f2da22e0 con 0x5615f2d8b480 2024-05-03T20:36:03.113 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.090+0000 7f3d2aff5700 1 -- 172.21.15.181:0/2217587690 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d40003c00 con 0x5615f2d8b480 2024-05-03T20:36:03.113 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.091+0000 7f3d2aff5700 1 -- 172.21.15.181:0/2217587690 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3d38006720 con 0x5615f2d8b480 2024-05-03T20:36:03.113 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.091+0000 7f3d57e96b80 10 client.15000 fetch_fsmap finished waiting for FSMap version 27 2024-05-03T20:36:03.113 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.091+0000 7f3d57e96b80 10 client.15000 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-03T20:36:03.113 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.091+0000 7f3d57e96b80 20 client.15000 populate_metadata read hostname 'smithi181' 2024-05-03T20:36:03.113 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.091+0000 7f3d57e96b80 1 --2- 172.21.15.181:0/2217587690 >> [v2:172.21.15.196:6838/1182826743,v1:172.21.15.196:6839/1182826743] conn(0x5615f2da4920 0x5615f2da5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:03.114 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.091+0000 7f3d57e96b80 4 client.15000 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T20:36:03.114 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.091+0000 7f3d57e96b80 1 -- 172.21.15.181:0/2217587690 --> [v2:172.21.15.196:6838/1182826743,v1:172.21.15.196:6839/1182826743] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615f2ceb2d0 con 0x5615f2da4920 2024-05-03T20:36:03.114 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.091+0000 7f3d467fc700 1 --2- 172.21.15.181:0/2217587690 >> [v2:172.21.15.196:6838/1182826743,v1:172.21.15.196:6839/1182826743] conn(0x5615f2da4920 0x5615f2da5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:03.114 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.091+0000 7f3d2aff5700 10 client.15000 ms_handle_connect on v2:172.21.15.196:6838/1182826743 2024-05-03T20:36:03.114 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.091+0000 7f3d467fc700 1 --2- 172.21.15.181:0/2217587690 >> [v2:172.21.15.196:6838/1182826743,v1:172.21.15.196:6839/1182826743] conn(0x5615f2da4920 0x5615f2da5d90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b14bf941a085547c server_cookie=3dd3a5dc2d0e89f5 in_seq=0 out_seq=0 2024-05-03T20:36:03.114 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.092+0000 7f3d2aff5700 1 -- 172.21.15.181:0/2217587690 <== mds.0 v2:172.21.15.196:6838/1182826743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5615f2ceb2d0 con 0x5615f2da4920 2024-05-03T20:36:03.114 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.092+0000 7f3d2aff5700 10 client.15000 handle_command_reply: tid=0 2024-05-03T20:36:03.115 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.092+0000 7f3d57e96b80 1 client.15000 shutdown 2024-05-03T20:36:03.115 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.092+0000 7f3d57e96b80 1 -- 172.21.15.181:0/2217587690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8b480 msgr2=0x5615f2d96d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:03.115 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.092+0000 7f3d57e96b80 1 --2- 172.21.15.181:0/2217587690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8b480 0x5615f2d96d20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3d380154f0 tx=0x7f3d38004860 comp rx=0 tx=0).stop 2024-05-03T20:36:03.115 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.092+0000 7f3d57e96b80 1 -- 172.21.15.181:0/2217587690 shutdown_connections 2024-05-03T20:36:03.115 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.092+0000 7f3d57e96b80 1 --2- 172.21.15.181:0/2217587690 >> [v2:172.21.15.196:6838/1182826743,v1:172.21.15.196:6839/1182826743] conn(0x5615f2da4920 0x5615f2da5d90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.115 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.092+0000 7f3d57e96b80 1 --2- 172.21.15.181:0/2217587690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615f2d90060 0x5615f2d94650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.116 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.092+0000 7f3d2aff5700 0 client.15000 ms_handle_reset on v2:172.21.15.196:6838/1182826743 2024-05-03T20:36:03.116 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.092+0000 7f3d57e96b80 1 --2- 172.21.15.181:0/2217587690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8b480 0x5615f2d96d20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.116 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.092+0000 7f3d57e96b80 1 --2- 172.21.15.181:0/2217587690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f2d8f740 0x5615f2d94110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.116 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.092+0000 7f3d57e96b80 1 -- 172.21.15.181:0/2217587690 >> 172.21.15.181:0/2217587690 conn(0x5615f2cb7b20 msgr2=0x5615f2d8e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:03.116 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.093+0000 7f3d57e96b80 1 -- 172.21.15.181:0/2217587690 shutdown_connections 2024-05-03T20:36:03.116 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.093+0000 7f3d57e96b80 1 -- 172.21.15.181:0/2217587690 wait complete. 2024-05-03T20:36:03.117 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.093+0000 7f3d57e96b80 20 client.15000 trim_cache size 0 max 16384 2024-05-03T20:36:03.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.114+0000 7f3d57e96b80 1 Processor -- start 2024-05-03T20:36:03.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.114+0000 7f3d57e96b80 1 -- start start 2024-05-03T20:36:03.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.114+0000 7f3d57e96b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f2d8b480 0x5615f2d85810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:03.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.114+0000 7f3d57e96b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8f740 0x5615f2d940d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:03.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.114+0000 7f3d57e96b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615f2d90060 0x5615f2d94610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:03.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.114+0000 7f3d57e96b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5615f2da40d0 con 0x5615f2d8f740 2024-05-03T20:36:03.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.114+0000 7f3d57e96b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5615f2da47a0 con 0x5615f2d8b480 2024-05-03T20:36:03.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.114+0000 7f3d57e96b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5615f2da4e70 con 0x5615f2d90060 2024-05-03T20:36:03.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.115+0000 7f3d467fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8f740 0x5615f2d940d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:03.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.115+0000 7f3d467fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8f740 0x5615f2d940d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58256/0 (socket says 172.21.15.181:58256) 2024-05-03T20:36:03.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.115+0000 7f3d467fc700 1 -- 172.21.15.181:0/3573648226 learned_addr learned my addr 172.21.15.181:0/3573648226 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:03.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.115+0000 7f3d46ffd700 1 --2- 172.21.15.181:0/3573648226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f2d8b480 0x5615f2d85810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:03.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.115+0000 7f3d467fc700 1 -- 172.21.15.181:0/3573648226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615f2d90060 msgr2=0x5615f2d94610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:03.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.115+0000 7f3d467fc700 1 --2- 172.21.15.181:0/3573648226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615f2d90060 0x5615f2d94610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.115+0000 7f3d467fc700 1 -- 172.21.15.181:0/3573648226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f2d8b480 msgr2=0x5615f2d85810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:03.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.115+0000 7f3d467fc700 1 --2- 172.21.15.181:0/3573648226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f2d8b480 0x5615f2d85810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.115+0000 7f3d467fc700 1 -- 172.21.15.181:0/3573648226 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615f2d8f390 con 0x5615f2d8f740 2024-05-03T20:36:03.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.115+0000 7f3d467fc700 1 --2- 172.21.15.181:0/3573648226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8f740 0x5615f2d940d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0a92d0 tx=0x7f3d3c004fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e130a68cfd71aa1 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:03.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.115+0000 7f3d2cff9700 1 -- 172.21.15.181:0/3573648226 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c0b2480 con 0x5615f2d8f740 2024-05-03T20:36:03.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.115+0000 7f3d2cff9700 1 -- 172.21.15.181:0/3573648226 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d3c0056e0 con 0x5615f2d8f740 2024-05-03T20:36:03.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.115+0000 7f3d2cff9700 1 -- 172.21.15.181:0/3573648226 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c0a9ae0 con 0x5615f2d8f740 2024-05-03T20:36:03.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.116+0000 7f3d57e96b80 1 -- 172.21.15.181:0/3573648226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8f740 msgr2=0x5615f2d940d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:03.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.116+0000 7f3d57e96b80 1 --2- 172.21.15.181:0/3573648226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8f740 0x5615f2d940d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0a92d0 tx=0x7f3d3c004fb0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.116+0000 7f3d57e96b80 1 -- 172.21.15.181:0/3573648226 shutdown_connections 2024-05-03T20:36:03.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.116+0000 7f3d57e96b80 1 --2- 172.21.15.181:0/3573648226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615f2d90060 0x5615f2d94610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.116+0000 7f3d57e96b80 1 --2- 172.21.15.181:0/3573648226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f2d8b480 0x5615f2d85810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.116+0000 7f3d57e96b80 1 --2- 172.21.15.181:0/3573648226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8f740 0x5615f2d940d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.116+0000 7f3d57e96b80 1 -- 172.21.15.181:0/3573648226 >> 172.21.15.181:0/3573648226 conn(0x5615f2cb7b20 msgr2=0x5615f2d8aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:03.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.116+0000 7f3d57e96b80 1 -- 172.21.15.181:0/3573648226 shutdown_connections 2024-05-03T20:36:03.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.116+0000 7f3d57e96b80 1 -- 172.21.15.181:0/3573648226 wait complete. 2024-05-03T20:36:03.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.117+0000 7f3d57e96b80 1 -- start start 2024-05-03T20:36:03.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.117+0000 7f3d57e96b80 1 Processor -- start 2024-05-03T20:36:03.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.117+0000 7f3d57e96b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8b480 0x5615f2da8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:03.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.117+0000 7f3d57e96b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615f2d8f740 0x5615f2da88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:03.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.117+0000 7f3d57e96b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f2d90060 0x5615f2dad270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:03.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.117+0000 7f3d57e96b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5615f2da5c10 con 0x5615f2d8b480 2024-05-03T20:36:03.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.117+0000 7f3d57e96b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5615f2da5540 con 0x5615f2d90060 2024-05-03T20:36:03.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.117+0000 7f3d57e96b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5615f2da62e0 con 0x5615f2d8f740 2024-05-03T20:36:03.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.117+0000 7f3d46ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8b480 0x5615f2da8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:03.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.117+0000 7f3d57e96b80 10 client.0 _ll_register_callbacks cb 0x5615f2c71560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T20:36:03.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.117+0000 7f3d46ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8b480 0x5615f2da8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58264/0 (socket says 172.21.15.181:58264) 2024-05-03T20:36:03.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.117+0000 7f3d46ffd700 1 -- 172.21.15.181:0/4142929026 learned_addr learned my addr 172.21.15.181:0/4142929026 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:03.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.117+0000 7f3d467fc700 1 --2- 172.21.15.181:0/4142929026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615f2d8f740 0x5615f2da88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:03.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.117+0000 7f3d477fe700 1 --2- 172.21.15.181:0/4142929026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f2d90060 0x5615f2dad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:03.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.117+0000 7f3d46ffd700 1 -- 172.21.15.181:0/4142929026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615f2d8f740 msgr2=0x5615f2da88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:03.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.117+0000 7f3d46ffd700 1 --2- 172.21.15.181:0/4142929026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615f2d8f740 0x5615f2da88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.117+0000 7f3d46ffd700 1 -- 172.21.15.181:0/4142929026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f2d90060 msgr2=0x5615f2dad270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:03.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.117+0000 7f3d46ffd700 1 --2- 172.21.15.181:0/4142929026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f2d90060 0x5615f2dad270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.117+0000 7f3d46ffd700 1 -- 172.21.15.181:0/4142929026 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615f2dad8e0 con 0x5615f2d8b480 2024-05-03T20:36:03.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.117+0000 7f3d467fc700 1 --2- 172.21.15.181:0/4142929026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615f2d8f740 0x5615f2da88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:36:03.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.118+0000 7f3d46ffd700 1 --2- 172.21.15.181:0/4142929026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8b480 0x5615f2da8390 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3d3800e530 tx=0x7f3d38022a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28a7ab698d6344b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:03.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.118+0000 7f3d2aff5700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-03T20:36:03.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.118+0000 7f3d2aff5700 1 -- 172.21.15.181:0/4142929026 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38025040 con 0x5615f2d8b480 2024-05-03T20:36:03.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.118+0000 7f3d2aff5700 1 -- 172.21.15.181:0/4142929026 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f3d38006240 con 0x5615f2d8b480 2024-05-03T20:36:03.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.118+0000 7f3d57e96b80 1 -- 172.21.15.181:0/4142929026 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d3c0056e0 con 0x5615f2d8b480 2024-05-03T20:36:03.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.118+0000 7f3d2aff5700 1 -- 172.21.15.181:0/4142929026 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d38015960 con 0x5615f2d8b480 2024-05-03T20:36:03.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.118+0000 7f3d57e96b80 10 client.15006 fetch_fsmap learned FSMap version 27 2024-05-03T20:36:03.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.118+0000 7f3d57e96b80 1 -- 172.21.15.181:0/4142929026 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5615f2daec50 con 0x5615f2d8b480 2024-05-03T20:36:03.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.119+0000 7f3d2aff5700 1 -- 172.21.15.181:0/4142929026 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d3c0056e0 con 0x5615f2d8b480 2024-05-03T20:36:03.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.119+0000 7f3d2aff5700 1 -- 172.21.15.181:0/4142929026 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3d380216b0 con 0x5615f2d8b480 2024-05-03T20:36:03.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.119+0000 7f3d57e96b80 10 client.15006 fetch_fsmap finished waiting for FSMap version 27 2024-05-03T20:36:03.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.119+0000 7f3d57e96b80 10 client.15006 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-03T20:36:03.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.119+0000 7f3d57e96b80 20 client.15006 populate_metadata read hostname 'smithi181' 2024-05-03T20:36:03.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.119+0000 7f3d57e96b80 1 --2- 172.21.15.181:0/4142929026 >> [v2:172.21.15.196:6838/1182826743,v1:172.21.15.196:6839/1182826743] conn(0x5615f2e15860 0x5615f2e14570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:03.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.119+0000 7f3d57e96b80 4 client.15006 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T20:36:03.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.119+0000 7f3d57e96b80 1 -- 172.21.15.181:0/4142929026 --> [v2:172.21.15.196:6838/1182826743,v1:172.21.15.196:6839/1182826743] -- command(tid 0: {"prefix": "status"}) -- 0x5615f2ce5370 con 0x5615f2e15860 2024-05-03T20:36:03.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.119+0000 7f3d467fc700 1 --2- 172.21.15.181:0/4142929026 >> [v2:172.21.15.196:6838/1182826743,v1:172.21.15.196:6839/1182826743] conn(0x5615f2e15860 0x5615f2e14570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:03.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.120+0000 7f3d467fc700 1 --2- 172.21.15.181:0/4142929026 >> [v2:172.21.15.196:6838/1182826743,v1:172.21.15.196:6839/1182826743] conn(0x5615f2e15860 0x5615f2e14570 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=880e62d9b0c03679 server_cookie=ebe65f351353686e in_seq=0 out_seq=0 2024-05-03T20:36:03.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.120+0000 7f3d2aff5700 10 client.15006 ms_handle_connect on v2:172.21.15.196:6838/1182826743 2024-05-03T20:36:03.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.120+0000 7f3d2aff5700 1 -- 172.21.15.181:0/4142929026 <== mds.0 v2:172.21.15.196:6838/1182826743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5615f2ce5370 con 0x5615f2e15860 2024-05-03T20:36:03.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.120+0000 7f3d2aff5700 10 client.15006 handle_command_reply: tid=0 2024-05-03T20:36:03.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.120+0000 7f3d57e96b80 1 client.15006 shutdown 2024-05-03T20:36:03.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.120+0000 7f3d57e96b80 1 -- 172.21.15.181:0/4142929026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8b480 msgr2=0x5615f2da8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:03.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.120+0000 7f3d57e96b80 1 --2- 172.21.15.181:0/4142929026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8b480 0x5615f2da8390 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3d3800e530 tx=0x7f3d38022a70 comp rx=0 tx=0).stop 2024-05-03T20:36:03.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.120+0000 7f3d57e96b80 1 -- 172.21.15.181:0/4142929026 shutdown_connections 2024-05-03T20:36:03.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.120+0000 7f3d57e96b80 1 --2- 172.21.15.181:0/4142929026 >> [v2:172.21.15.196:6838/1182826743,v1:172.21.15.196:6839/1182826743] conn(0x5615f2e15860 0x5615f2e14570 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.120+0000 7f3d57e96b80 1 --2- 172.21.15.181:0/4142929026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f2d90060 0x5615f2dad270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.120+0000 7f3d57e96b80 1 --2- 172.21.15.181:0/4142929026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5615f2d8b480 0x5615f2da8390 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.120+0000 7f3d57e96b80 1 --2- 172.21.15.181:0/4142929026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5615f2d8f740 0x5615f2da88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.120+0000 7f3d57e96b80 1 -- 172.21.15.181:0/4142929026 >> 172.21.15.181:0/4142929026 conn(0x5615f2cb7b20 msgr2=0x5615f2d940d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:03.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.120+0000 7f3d2aff5700 0 client.15006 ms_handle_reset on v2:172.21.15.196:6838/1182826743 2024-05-03T20:36:03.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.120+0000 7f3d57e96b80 1 -- 172.21.15.181:0/4142929026 shutdown_connections 2024-05-03T20:36:03.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.121+0000 7f3d57e96b80 1 -- 172.21.15.181:0/4142929026 wait complete. 2024-05-03T20:36:03.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.121+0000 7f3d57e96b80 20 client.15006 trim_cache size 0 max 16384 2024-05-03T20:36:03.129 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-05-03T20:36:03.129 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "1cd058ee-098b-11ef-bc97-c7b262605968", 2024-05-03T20:36:03.129 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 0, 2024-05-03T20:36:03.129 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 24488, 2024-05-03T20:36:03.129 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2024-05-03T20:36:03.129 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2024-05-03T20:36:03.129 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2024-05-03T20:36:03.129 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 140.41674474199999, 2024-05-03T20:36:03.129 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 27, 2024-05-03T20:36:03.129 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 83, 2024-05-03T20:36:03.129 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 81, 2024-05-03T20:36:03.129 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 154.91705073700001 2024-05-03T20:36:03.129 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-05-03T20:36:03.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.121+0000 7f3d4d603700 1 -- 172.21.15.181:0/3329801765 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3d300b2910 msgr2=0x7f3d300b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:03.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.121+0000 7f3d4d603700 1 --2- 172.21.15.181:0/3329801765 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3d300b2910 0x7f3d300b4dd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3d38009a10 tx=0x7f3d38009190 comp rx=0 tx=0).stop 2024-05-03T20:36:03.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.121+0000 7f3d4d603700 1 -- 172.21.15.181:0/3329801765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d48103eb0 msgr2=0x7f3d48075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:03.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.121+0000 7f3d4d603700 1 --2- 172.21.15.181:0/3329801765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d48103eb0 0x7f3d48075ce0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00d980 tx=0x7f3d3c00dc90 comp rx=0 tx=0).stop 2024-05-03T20:36:03.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.122+0000 7f3d4d603700 1 -- 172.21.15.181:0/3329801765 shutdown_connections 2024-05-03T20:36:03.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.122+0000 7f3d4d603700 1 --2- 172.21.15.181:0/3329801765 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3d300b2910 0x7f3d300b4dd0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.122+0000 7f3d4d603700 1 --2- 172.21.15.181:0/3329801765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d48109600 0x7f3d48076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.122+0000 7f3d4d603700 1 --2- 172.21.15.181:0/3329801765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d481034f0 0x7f3d480757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.122+0000 7f3d4d603700 1 --2- 172.21.15.181:0/3329801765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d48103eb0 0x7f3d48075ce0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:03.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.122+0000 7f3d4d603700 1 -- 172.21.15.181:0/3329801765 >> 172.21.15.181:0/3329801765 conn(0x7f3d4807a650 msgr2=0x7f3d48101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:03.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.122+0000 7f3d4d603700 1 -- 172.21.15.181:0/3329801765 shutdown_connections 2024-05-03T20:36:03.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:03.122+0000 7f3d4d603700 1 -- 172.21.15.181:0/3329801765 wait complete. 2024-05-03T20:36:03.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:03 smithi181 ceph-mon[29801]: pgmap v357: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:36:03.777 INFO:tasks.cephadm:Setup complete, yielding 2024-05-03T20:36:03.777 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-03T20:36:03.792 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi181.front.sepia.ceph.com 2024-05-03T20:36:03.792 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-03T20:36:03.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:03 smithi196 ceph-mon[32858]: pgmap v357: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:36:03.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:03 smithi204 ceph-mon[33595]: pgmap v357: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:36:04.065 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:36:05.177 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:04 smithi181 ceph-mon[29801]: pgmap v358: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:36:05.177 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:04 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:36:05.178 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:04 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:36:05.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:04 smithi196 ceph-mon[32858]: pgmap v358: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:36:05.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:04 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:36:05.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:04 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:36:05.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:04 smithi204 ceph-mon[33595]: pgmap v358: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:36:05.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:04 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:36:05.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:04 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:36:05.700 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.698+0000 7fe29b0ba700 1 -- 172.21.15.181:0/3899243075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2941050c0 msgr2=0x7fe29410b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:05.701 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.698+0000 7fe29b0ba700 1 --2- 172.21.15.181:0/3899243075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2941050c0 0x7fe29410b970 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe2840099c0 tx=0x7fe284009cd0 comp rx=0 tx=0).stop 2024-05-03T20:36:05.701 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.698+0000 7fe29b0ba700 1 -- 172.21.15.181:0/3899243075 shutdown_connections 2024-05-03T20:36:05.701 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.698+0000 7fe29b0ba700 1 --2- 172.21.15.181:0/3899243075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2941050c0 0x7fe29410b970 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:05.701 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.698+0000 7fe29b0ba700 1 --2- 172.21.15.181:0/3899243075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe294103520 0x7fe294103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:05.701 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.698+0000 7fe29b0ba700 1 --2- 172.21.15.181:0/3899243075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe294104720 0x7fe294104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:05.701 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.698+0000 7fe29b0ba700 1 -- 172.21.15.181:0/3899243075 >> 172.21.15.181:0/3899243075 conn(0x7fe2940feac0 msgr2=0x7fe294100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:05.701 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.698+0000 7fe29b0ba700 1 -- 172.21.15.181:0/3899243075 shutdown_connections 2024-05-03T20:36:05.701 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.698+0000 7fe29b0ba700 1 -- 172.21.15.181:0/3899243075 wait complete. 2024-05-03T20:36:05.701 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.699+0000 7fe29b0ba700 1 Processor -- start 2024-05-03T20:36:05.702 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.699+0000 7fe29b0ba700 1 -- start start 2024-05-03T20:36:05.702 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.699+0000 7fe29b0ba700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe294103520 0x7fe2941995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:05.702 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.699+0000 7fe29b0ba700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe294104720 0x7fe294199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:05.702 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.699+0000 7fe29b0ba700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2941050c0 0x7fe2941a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:05.702 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.699+0000 7fe29b0ba700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe29410f040 con 0x7fe294103520 2024-05-03T20:36:05.702 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.699+0000 7fe29b0ba700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe29410eec0 con 0x7fe294104720 2024-05-03T20:36:05.702 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.699+0000 7fe29b0ba700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe29410f1c0 con 0x7fe2941050c0 2024-05-03T20:36:05.702 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.699+0000 7fe298e56700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe294103520 0x7fe2941995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:05.702 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.699+0000 7fe298e56700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe294103520 0x7fe2941995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42552/0 (socket says 172.21.15.181:42552) 2024-05-03T20:36:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.699+0000 7fe298e56700 1 -- 172.21.15.181:0/2007603536 learned_addr learned my addr 172.21.15.181:0/2007603536 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.699+0000 7fe293fff700 1 --2- 172.21.15.181:0/2007603536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe294104720 0x7fe294199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.700+0000 7fe299657700 1 --2- 172.21.15.181:0/2007603536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2941050c0 0x7fe2941a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.700+0000 7fe298e56700 1 -- 172.21.15.181:0/2007603536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2941050c0 msgr2=0x7fe2941a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.700+0000 7fe298e56700 1 --2- 172.21.15.181:0/2007603536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2941050c0 0x7fe2941a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.700+0000 7fe298e56700 1 -- 172.21.15.181:0/2007603536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe294104720 msgr2=0x7fe294199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.700+0000 7fe298e56700 1 --2- 172.21.15.181:0/2007603536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe294104720 0x7fe294199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.700+0000 7fe298e56700 1 -- 172.21.15.181:0/2007603536 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2941a1250 con 0x7fe294103520 2024-05-03T20:36:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.700+0000 7fe298e56700 1 --2- 172.21.15.181:0/2007603536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe294103520 0x7fe2941995f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe28c00b9f0 tx=0x7fe28c00bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.700+0000 7fe291ffb700 1 -- 172.21.15.181:0/2007603536 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c019070 con 0x7fe294103520 2024-05-03T20:36:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.700+0000 7fe291ffb700 1 -- 172.21.15.181:0/2007603536 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe28c00ce00 con 0x7fe294103520 2024-05-03T20:36:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.700+0000 7fe29b0ba700 1 -- 172.21.15.181:0/2007603536 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe294075730 con 0x7fe294103520 2024-05-03T20:36:05.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.700+0000 7fe291ffb700 1 -- 172.21.15.181:0/2007603536 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c012340 con 0x7fe294103520 2024-05-03T20:36:05.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.700+0000 7fe29b0ba700 1 -- 172.21.15.181:0/2007603536 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe294075c20 con 0x7fe294103520 2024-05-03T20:36:05.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.701+0000 7fe29b0ba700 1 -- 172.21.15.181:0/2007603536 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe29406a190 con 0x7fe294103520 2024-05-03T20:36:05.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.702+0000 7fe291ffb700 1 -- 172.21.15.181:0/2007603536 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fe28c01c070 con 0x7fe294103520 2024-05-03T20:36:05.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.703+0000 7fe291ffb700 1 --2- 172.21.15.181:0/2007603536 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fe2800b2910 0x7fe2800b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:05.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.703+0000 7fe293fff700 1 --2- 172.21.15.181:0/2007603536 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fe2800b2910 0x7fe2800b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:05.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.703+0000 7fe291ffb700 1 -- 172.21.15.181:0/2007603536 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7fe28c0e16b0 con 0x7fe294103520 2024-05-03T20:36:05.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.703+0000 7fe293fff700 1 --2- 172.21.15.181:0/2007603536 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fe2800b2910 0x7fe2800b4dd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe27c005950 tx=0x7fe27c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:05.707 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.708+0000 7fe291ffb700 1 -- 172.21.15.181:0/2007603536 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe28c0a9380 con 0x7fe294103520 2024-05-03T20:36:05.935 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.936+0000 7fe29b0ba700 1 -- 172.21.15.181:0/2007603536 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fe2940638e0 con 0x7fe2800b2910 2024-05-03T20:36:05.936 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.937+0000 7fe291ffb700 1 -- 172.21.15.181:0/2007603536 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fe2940638e0 con 0x7fe2800b2910 2024-05-03T20:36:05.936 INFO:teuthology.orchestra.run.smithi181.stdout:Backend: cephadm 2024-05-03T20:36:05.936 INFO:teuthology.orchestra.run.smithi181.stdout:Available: Yes 2024-05-03T20:36:05.936 INFO:teuthology.orchestra.run.smithi181.stdout:Paused: No 2024-05-03T20:36:05.940 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.939+0000 7fe29b0ba700 1 -- 172.21.15.181:0/2007603536 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fe2800b2910 msgr2=0x7fe2800b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:05.940 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.939+0000 7fe29b0ba700 1 --2- 172.21.15.181:0/2007603536 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fe2800b2910 0x7fe2800b4dd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe27c005950 tx=0x7fe27c00a300 comp rx=0 tx=0).stop 2024-05-03T20:36:05.940 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.939+0000 7fe29b0ba700 1 -- 172.21.15.181:0/2007603536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe294103520 msgr2=0x7fe2941995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:05.940 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.939+0000 7fe29b0ba700 1 --2- 172.21.15.181:0/2007603536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe294103520 0x7fe2941995f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe28c00b9f0 tx=0x7fe28c00bd00 comp rx=0 tx=0).stop 2024-05-03T20:36:05.940 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.939+0000 7fe29b0ba700 1 -- 172.21.15.181:0/2007603536 shutdown_connections 2024-05-03T20:36:05.940 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.939+0000 7fe29b0ba700 1 --2- 172.21.15.181:0/2007603536 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fe2800b2910 0x7fe2800b4dd0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:05.940 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.939+0000 7fe29b0ba700 1 --2- 172.21.15.181:0/2007603536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2941050c0 0x7fe2941a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:05.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.939+0000 7fe29b0ba700 1 --2- 172.21.15.181:0/2007603536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe294103520 0x7fe2941995f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:05.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.939+0000 7fe29b0ba700 1 --2- 172.21.15.181:0/2007603536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe294104720 0x7fe294199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:05.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.939+0000 7fe29b0ba700 1 -- 172.21.15.181:0/2007603536 >> 172.21.15.181:0/2007603536 conn(0x7fe2940feac0 msgr2=0x7fe294107a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:05.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.940+0000 7fe29b0ba700 1 -- 172.21.15.181:0/2007603536 shutdown_connections 2024-05-03T20:36:05.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:05.940+0000 7fe29b0ba700 1 -- 172.21.15.181:0/2007603536 wait complete. 2024-05-03T20:36:06.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:05 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:36:06.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:05 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:36:06.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:05 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:36:06.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:05 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:36:06.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:05 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:36:06.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:05 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:36:06.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:05 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:36:06.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:05 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:36:06.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:05 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:36:06.629 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-03T20:36:06.869 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:36:07.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:07 smithi196 ceph-mon[32858]: pgmap v359: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:36:07.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:07 smithi196 ceph-mon[32858]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:36:07.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:07 smithi204 ceph-mon[33595]: pgmap v359: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:36:07.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:07 smithi204 ceph-mon[33595]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:36:07.632 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:07 smithi181 ceph-mon[29801]: pgmap v359: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:36:07.633 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:07 smithi181 ceph-mon[29801]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:36:08.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.423+0000 7f0213156700 1 -- 172.21.15.181:0/2416945855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f020c102dd0 msgr2=0x7f020c103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:08.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.423+0000 7f0213156700 1 --2- 172.21.15.181:0/2416945855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f020c102dd0 0x7f020c103230 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f01f80099c0 tx=0x7f01f8009cd0 comp rx=0 tx=0).stop 2024-05-03T20:36:08.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.423+0000 7f0213156700 1 -- 172.21.15.181:0/2416945855 shutdown_connections 2024-05-03T20:36:08.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.423+0000 7f0213156700 1 --2- 172.21.15.181:0/2416945855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f020c103770 0x7f020c10a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:08.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.423+0000 7f0213156700 1 --2- 172.21.15.181:0/2416945855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f020c101bd0 0x7f020c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:08.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.423+0000 7f0213156700 1 --2- 172.21.15.181:0/2416945855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f020c102dd0 0x7f020c103230 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:08.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.423+0000 7f0213156700 1 -- 172.21.15.181:0/2416945855 >> 172.21.15.181:0/2416945855 conn(0x7f020c0fd170 msgr2=0x7f020c0ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:08.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.423+0000 7f0213156700 1 -- 172.21.15.181:0/2416945855 shutdown_connections 2024-05-03T20:36:08.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.423+0000 7f0213156700 1 -- 172.21.15.181:0/2416945855 wait complete. 2024-05-03T20:36:08.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.424+0000 7f0213156700 1 Processor -- start 2024-05-03T20:36:08.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.424+0000 7f0213156700 1 -- start start 2024-05-03T20:36:08.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.424+0000 7f0213156700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f020c101bd0 0x7f020c10d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:08.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.424+0000 7f0213156700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f020c102dd0 0x7f020c19e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:08.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.424+0000 7f0213156700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f020c103770 0x7f020c1a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:08.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.424+0000 7f0213156700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f020c10ded0 con 0x7f020c101bd0 2024-05-03T20:36:08.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.424+0000 7f0213156700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f020c10dd50 con 0x7f020c103770 2024-05-03T20:36:08.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.424+0000 7f0213156700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f020c10e050 con 0x7f020c102dd0 2024-05-03T20:36:08.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.424+0000 7f0210ef2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f020c101bd0 0x7f020c10d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:08.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.424+0000 7f0210ef2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f020c101bd0 0x7f020c10d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42570/0 (socket says 172.21.15.181:42570) 2024-05-03T20:36:08.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.424+0000 7f0210ef2700 1 -- 172.21.15.181:0/394206096 learned_addr learned my addr 172.21.15.181:0/394206096 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:08.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.424+0000 7f020bfff700 1 --2- 172.21.15.181:0/394206096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f020c102dd0 0x7f020c19e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:08.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.424+0000 7f0210ef2700 1 -- 172.21.15.181:0/394206096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f020c102dd0 msgr2=0x7f020c19e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:08.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.424+0000 7f0210ef2700 1 --2- 172.21.15.181:0/394206096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f020c102dd0 0x7f020c19e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:08.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.424+0000 7f02116f3700 1 --2- 172.21.15.181:0/394206096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f020c103770 0x7f020c1a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:08.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.424+0000 7f0210ef2700 1 -- 172.21.15.181:0/394206096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f020c103770 msgr2=0x7f020c1a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:08.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.424+0000 7f0210ef2700 1 --2- 172.21.15.181:0/394206096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f020c103770 0x7f020c1a0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:08.427 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.424+0000 7f0210ef2700 1 -- 172.21.15.181:0/394206096 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020c1a1260 con 0x7f020c101bd0 2024-05-03T20:36:08.427 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.424+0000 7f0210ef2700 1 --2- 172.21.15.181:0/394206096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f020c101bd0 0x7f020c10d7b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f020000c990 tx=0x7f020000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:08.427 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.425+0000 7f0209ffb700 1 -- 172.21.15.181:0/394206096 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0200007b00 con 0x7f020c101bd0 2024-05-03T20:36:08.427 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.425+0000 7f0209ffb700 1 -- 172.21.15.181:0/394206096 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0200007ca0 con 0x7f020c101bd0 2024-05-03T20:36:08.427 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.425+0000 7f0213156700 1 -- 172.21.15.181:0/394206096 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f020c1a1550 con 0x7f020c101bd0 2024-05-03T20:36:08.427 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.425+0000 7f0209ffb700 1 -- 172.21.15.181:0/394206096 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0200018660 con 0x7f020c101bd0 2024-05-03T20:36:08.427 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.425+0000 7f0213156700 1 -- 172.21.15.181:0/394206096 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f020c1a1a90 con 0x7f020c101bd0 2024-05-03T20:36:08.427 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.426+0000 7f0213156700 1 -- 172.21.15.181:0/394206096 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f020c06a190 con 0x7f020c101bd0 2024-05-03T20:36:08.427 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.428+0000 7f0209ffb700 1 -- 172.21.15.181:0/394206096 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f0200004020 con 0x7f020c101bd0 2024-05-03T20:36:08.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.429+0000 7f0209ffb700 1 --2- 172.21.15.181:0/394206096 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f01fc0b2910 0x7f01fc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:08.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.429+0000 7f020bfff700 1 --2- 172.21.15.181:0/394206096 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f01fc0b2910 0x7f01fc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:08.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.429+0000 7f0209ffb700 1 -- 172.21.15.181:0/394206096 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f02000e2630 con 0x7f020c101bd0 2024-05-03T20:36:08.429 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.429+0000 7f020bfff700 1 --2- 172.21.15.181:0/394206096 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f01fc0b2910 0x7f01fc0b4dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f01f8006010 tx=0x7f01f8009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:08.431 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.432+0000 7f0209ffb700 1 -- 172.21.15.181:0/394206096 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0200014030 con 0x7f020c101bd0 2024-05-03T20:36:08.645 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.645+0000 7f0213156700 1 -- 172.21.15.181:0/394206096 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f020c0638e0 con 0x7f01fc0b2910 2024-05-03T20:36:08.658 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.659+0000 7f0209ffb700 1 -- 172.21.15.181:0/394206096 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f020c0638e0 con 0x7f01fc0b2910 2024-05-03T20:36:08.658 INFO:teuthology.orchestra.run.smithi181.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-03T20:36:08.659 INFO:teuthology.orchestra.run.smithi181.stdout:mds.a smithi181 running (2m) 4s ago 2m 21.5M - 19.0.0-3450-g9a047fe7 ee4d84620249 0743a3767730 2024-05-03T20:36:08.659 INFO:teuthology.orchestra.run.smithi181.stdout:mds.b smithi196 running (2m) 7s ago 2m 29.0M - 19.0.0-3450-g9a047fe7 ee4d84620249 0cc7fb0361cd 2024-05-03T20:36:08.659 INFO:teuthology.orchestra.run.smithi181.stdout:mds.c smithi204 running (2m) 7s ago 2m 19.2M - 19.0.0-3450-g9a047fe7 ee4d84620249 df2ea6b1690e 2024-05-03T20:36:08.659 INFO:teuthology.orchestra.run.smithi181.stdout:mds.d smithi181 running (3m) 4s ago 3m 14.7M - 19.0.0-3450-g9a047fe7 ee4d84620249 50f687277cd5 2024-05-03T20:36:08.659 INFO:teuthology.orchestra.run.smithi181.stdout:mds.e smithi196 running (3m) 7s ago 3m 22.7M - 19.0.0-3450-g9a047fe7 ee4d84620249 cb61e4c3e20c 2024-05-03T20:36:08.659 INFO:teuthology.orchestra.run.smithi181.stdout:mds.f smithi204 running (2m) 7s ago 2m 19.5M - 19.0.0-3450-g9a047fe7 ee4d84620249 a4aca1db7f9d 2024-05-03T20:36:08.659 INFO:teuthology.orchestra.run.smithi181.stdout:mds.g smithi181 running (2m) 4s ago 2m 22.4M - 19.0.0-3450-g9a047fe7 ee4d84620249 d37b47955bc7 2024-05-03T20:36:08.659 INFO:teuthology.orchestra.run.smithi181.stdout:mds.h smithi196 running (2m) 7s ago 2m 21.7M - 19.0.0-3450-g9a047fe7 ee4d84620249 2f6a0b1b3e94 2024-05-03T20:36:08.659 INFO:teuthology.orchestra.run.smithi181.stdout:mds.i smithi204 running (2m) 7s ago 2m 20.2M - 19.0.0-3450-g9a047fe7 ee4d84620249 23106480b45a 2024-05-03T20:36:08.659 INFO:teuthology.orchestra.run.smithi181.stdout:mds.j smithi181 running (3m) 4s ago 3m 13.6M - 19.0.0-3450-g9a047fe7 ee4d84620249 37665f477d84 2024-05-03T20:36:08.659 INFO:teuthology.orchestra.run.smithi181.stdout:mds.k smithi196 running (3m) 7s ago 3m 20.2M - 19.0.0-3450-g9a047fe7 ee4d84620249 dcb17aadb68d 2024-05-03T20:36:08.659 INFO:teuthology.orchestra.run.smithi181.stdout:mds.l smithi204 running (3m) 7s ago 3m 20.1M - 19.0.0-3450-g9a047fe7 ee4d84620249 271a1961e502 2024-05-03T20:36:08.659 INFO:teuthology.orchestra.run.smithi181.stdout:mgr.x smithi181 *:9283,8765 running (10m) 4s ago 10m 529M - 19.0.0-3450-g9a047fe7 ee4d84620249 87c0517d69b1 2024-05-03T20:36:08.659 INFO:teuthology.orchestra.run.smithi181.stdout:mgr.y smithi196 *:8443,8765 running (8m) 7s ago 8m 466M - 19.0.0-3450-g9a047fe7 ee4d84620249 fba3b2277bf8 2024-05-03T20:36:08.659 INFO:teuthology.orchestra.run.smithi181.stdout:mgr.z smithi204 *:8443,8765 running (8m) 7s ago 8m 466M - 19.0.0-3450-g9a047fe7 ee4d84620249 a0992a0847c6 2024-05-03T20:36:08.660 INFO:teuthology.orchestra.run.smithi181.stdout:mon.a smithi181 running (10m) 4s ago 10m 53.6M 2048M 19.0.0-3450-g9a047fe7 ee4d84620249 e1d7a382742a 2024-05-03T20:36:08.660 INFO:teuthology.orchestra.run.smithi181.stdout:mon.b smithi196 running (8m) 7s ago 8m 43.3M 2048M 19.0.0-3450-g9a047fe7 ee4d84620249 d34180c69a8f 2024-05-03T20:36:08.660 INFO:teuthology.orchestra.run.smithi181.stdout:mon.c smithi204 running (8m) 7s ago 8m 40.0M 2048M 19.0.0-3450-g9a047fe7 ee4d84620249 f777a02cfbdf 2024-05-03T20:36:08.660 INFO:teuthology.orchestra.run.smithi181.stdout:osd.0 smithi181 running (8m) 4s ago 8m 59.4M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 b6ac61757097 2024-05-03T20:36:08.660 INFO:teuthology.orchestra.run.smithi181.stdout:osd.1 smithi196 running (7m) 7s ago 7m 57.3M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 f7b280b00d60 2024-05-03T20:36:08.660 INFO:teuthology.orchestra.run.smithi181.stdout:osd.2 smithi204 running (7m) 7s ago 7m 60.2M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 36dd222b9586 2024-05-03T20:36:08.660 INFO:teuthology.orchestra.run.smithi181.stdout:osd.3 smithi181 running (6m) 4s ago 6m 58.3M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 dd1a437cd7b2 2024-05-03T20:36:08.660 INFO:teuthology.orchestra.run.smithi181.stdout:osd.4 smithi196 running (6m) 7s ago 6m 48.8M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 b193e77ecca8 2024-05-03T20:36:08.660 INFO:teuthology.orchestra.run.smithi181.stdout:osd.5 smithi204 running (6m) 7s ago 6m 46.9M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 3728539110b8 2024-05-03T20:36:08.660 INFO:teuthology.orchestra.run.smithi181.stdout:osd.6 smithi181 running (5m) 4s ago 5m 45.6M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 0a7f20400d87 2024-05-03T20:36:08.660 INFO:teuthology.orchestra.run.smithi181.stdout:osd.7 smithi196 running (5m) 7s ago 5m 48.5M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 94c47cd86dc7 2024-05-03T20:36:08.660 INFO:teuthology.orchestra.run.smithi181.stdout:osd.8 smithi204 running (4m) 7s ago 4m 56.1M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 ab3c3d9c2e60 2024-05-03T20:36:08.660 INFO:teuthology.orchestra.run.smithi181.stdout:osd.9 smithi181 running (4m) 4s ago 4m 52.5M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 774db69cb90a 2024-05-03T20:36:08.660 INFO:teuthology.orchestra.run.smithi181.stdout:osd.10 smithi196 running (4m) 7s ago 4m 51.2M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 3022140c2af4 2024-05-03T20:36:08.661 INFO:teuthology.orchestra.run.smithi181.stdout:osd.11 smithi204 running (3m) 7s ago 3m 47.2M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 be95d51f72ca 2024-05-03T20:36:08.661 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.661+0000 7f0213156700 1 -- 172.21.15.181:0/394206096 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f01fc0b2910 msgr2=0x7f01fc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:08.661 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.661+0000 7f0213156700 1 --2- 172.21.15.181:0/394206096 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f01fc0b2910 0x7f01fc0b4dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f01f8006010 tx=0x7f01f8009f10 comp rx=0 tx=0).stop 2024-05-03T20:36:08.661 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.661+0000 7f0213156700 1 -- 172.21.15.181:0/394206096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f020c101bd0 msgr2=0x7f020c10d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:08.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.661+0000 7f0213156700 1 --2- 172.21.15.181:0/394206096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f020c101bd0 0x7f020c10d7b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f020000c990 tx=0x7f020000cd50 comp rx=0 tx=0).stop 2024-05-03T20:36:08.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.661+0000 7f0213156700 1 -- 172.21.15.181:0/394206096 shutdown_connections 2024-05-03T20:36:08.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.661+0000 7f0213156700 1 --2- 172.21.15.181:0/394206096 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f01fc0b2910 0x7f01fc0b4dd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:08.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.661+0000 7f0213156700 1 --2- 172.21.15.181:0/394206096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f020c103770 0x7f020c1a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:08.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.661+0000 7f0213156700 1 --2- 172.21.15.181:0/394206096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f020c101bd0 0x7f020c10d7b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:08.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.661+0000 7f0213156700 1 --2- 172.21.15.181:0/394206096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f020c102dd0 0x7f020c19e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:08.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.661+0000 7f0213156700 1 -- 172.21.15.181:0/394206096 >> 172.21.15.181:0/394206096 conn(0x7f020c0fd170 msgr2=0x7f020c106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:08.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.661+0000 7f0213156700 1 -- 172.21.15.181:0/394206096 shutdown_connections 2024-05-03T20:36:08.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:08.662+0000 7f0213156700 1 -- 172.21.15.181:0/394206096 wait complete. 2024-05-03T20:36:09.326 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-03T20:36:09.573 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:36:09.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:09 smithi196 ceph-mon[32858]: pgmap v360: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:36:09.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:09 smithi196 ceph-mon[32858]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:36:09.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:09 smithi204 ceph-mon[33595]: pgmap v360: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:36:09.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:09 smithi204 ceph-mon[33595]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:36:09.695 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:09 smithi181 ceph-mon[29801]: pgmap v360: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:36:09.695 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:09 smithi181 ceph-mon[29801]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:36:11.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.180+0000 7f4ee85ef700 1 -- 172.21.15.181:0/4239716256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ee01048b0 msgr2=0x7f4ee0104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:11.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.180+0000 7f4ee85ef700 1 --2- 172.21.15.181:0/4239716256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ee01048b0 0x7f4ee0104d10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00099c0 tx=0x7f4ed0009cd0 comp rx=0 tx=0).stop 2024-05-03T20:36:11.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.180+0000 7f4ee85ef700 1 -- 172.21.15.181:0/4239716256 shutdown_connections 2024-05-03T20:36:11.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.180+0000 7f4ee85ef700 1 --2- 172.21.15.181:0/4239716256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ee0105250 0x7f4ee010bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:11.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.180+0000 7f4ee85ef700 1 --2- 172.21.15.181:0/4239716256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ee0101830 0x7f4ee0101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:11.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.180+0000 7f4ee85ef700 1 --2- 172.21.15.181:0/4239716256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ee01048b0 0x7f4ee0104d10 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:11.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.180+0000 7f4ee85ef700 1 -- 172.21.15.181:0/4239716256 >> 172.21.15.181:0/4239716256 conn(0x7f4ee00fd170 msgr2=0x7f4ee00ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:11.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.180+0000 7f4ee85ef700 1 -- 172.21.15.181:0/4239716256 shutdown_connections 2024-05-03T20:36:11.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.180+0000 7f4ee85ef700 1 -- 172.21.15.181:0/4239716256 wait complete. 2024-05-03T20:36:11.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.180+0000 7f4ee85ef700 1 Processor -- start 2024-05-03T20:36:11.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.181+0000 7f4ee85ef700 1 -- start start 2024-05-03T20:36:11.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.181+0000 7f4ee85ef700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ee0101830 0x7f4ee0104210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:11.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.181+0000 7f4ee85ef700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ee01048b0 0x7f4ee0102860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:11.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.181+0000 7f4ee85ef700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ee0105250 0x7f4ee0102da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:11.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.181+0000 7f4ee85ef700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee010f010 con 0x7f4ee0105250 2024-05-03T20:36:11.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.181+0000 7f4ee85ef700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee010ee90 con 0x7f4ee01048b0 2024-05-03T20:36:11.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.181+0000 7f4ee85ef700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee010f190 con 0x7f4ee0101830 2024-05-03T20:36:11.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.181+0000 7f4ee6b8c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ee0105250 0x7f4ee0102da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:11.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.181+0000 7f4ee6b8c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ee0105250 0x7f4ee0102da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42586/0 (socket says 172.21.15.181:42586) 2024-05-03T20:36:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.181+0000 7f4ee6b8c700 1 -- 172.21.15.181:0/3269078946 learned_addr learned my addr 172.21.15.181:0/3269078946 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.181+0000 7f4ee5b8a700 1 --2- 172.21.15.181:0/3269078946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ee01048b0 0x7f4ee0102860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.181+0000 7f4ee638b700 1 --2- 172.21.15.181:0/3269078946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ee0101830 0x7f4ee0104210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.181+0000 7f4ee6b8c700 1 -- 172.21.15.181:0/3269078946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ee0101830 msgr2=0x7f4ee0104210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.181+0000 7f4ee6b8c700 1 --2- 172.21.15.181:0/3269078946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ee0101830 0x7f4ee0104210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.181+0000 7f4ee6b8c700 1 -- 172.21.15.181:0/3269078946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ee01048b0 msgr2=0x7f4ee0102860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.181+0000 7f4ee6b8c700 1 --2- 172.21.15.181:0/3269078946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ee01048b0 0x7f4ee0102860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.181+0000 7f4ee6b8c700 1 -- 172.21.15.181:0/3269078946 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee0103650 con 0x7f4ee0105250 2024-05-03T20:36:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.181+0000 7f4ee6b8c700 1 --2- 172.21.15.181:0/3269078946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ee0105250 0x7f4ee0102da0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8007ee0 tx=0x7f4ed800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.181+0000 7f4ed77fe700 1 -- 172.21.15.181:0/3269078946 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed800dbd0 con 0x7f4ee0105250 2024-05-03T20:36:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.181+0000 7f4ed77fe700 1 -- 172.21.15.181:0/3269078946 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ed800dd70 con 0x7f4ee0105250 2024-05-03T20:36:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.181+0000 7f4ee85ef700 1 -- 172.21.15.181:0/3269078946 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee01a3780 con 0x7f4ee0105250 2024-05-03T20:36:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.182+0000 7f4ed77fe700 1 -- 172.21.15.181:0/3269078946 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8005990 con 0x7f4ee0105250 2024-05-03T20:36:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.182+0000 7f4ee85ef700 1 -- 172.21.15.181:0/3269078946 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee01a3c40 con 0x7f4ee0105250 2024-05-03T20:36:11.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.182+0000 7f4ee85ef700 1 -- 172.21.15.181:0/3269078946 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee006a190 con 0x7f4ee0105250 2024-05-03T20:36:11.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.184+0000 7f4ed77fe700 1 -- 172.21.15.181:0/3269078946 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f4ed801a020 con 0x7f4ee0105250 2024-05-03T20:36:11.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.184+0000 7f4ed77fe700 1 --2- 172.21.15.181:0/3269078946 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4ecc0b2910 0x7f4ecc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:11.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.184+0000 7f4ee638b700 1 --2- 172.21.15.181:0/3269078946 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4ecc0b2910 0x7f4ecc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:11.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.184+0000 7f4ed77fe700 1 -- 172.21.15.181:0/3269078946 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f4ed80e2230 con 0x7f4ee0105250 2024-05-03T20:36:11.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.185+0000 7f4ee638b700 1 --2- 172.21.15.181:0/3269078946 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4ecc0b2910 0x7f4ecc0b4dd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4edc005fd0 tx=0x7f4edc005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:11.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.188+0000 7f4ed77fe700 1 -- 172.21.15.181:0/3269078946 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4ed8010070 con 0x7f4ee0105250 2024-05-03T20:36:11.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.404+0000 7f4ee85ef700 1 -- 172.21.15.181:0/3269078946 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f4ee00638e0 con 0x7f4ecc0b2910 2024-05-03T20:36:11.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.410+0000 7f4ed77fe700 1 -- 172.21.15.181:0/3269078946 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f4ee00638e0 con 0x7f4ecc0b2910 2024-05-03T20:36:11.409 INFO:teuthology.orchestra.run.smithi181.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-03T20:36:11.409 INFO:teuthology.orchestra.run.smithi181.stdout:mds.all 12/12 10s ago 3m smithi181=a;smithi181=d;smithi181=g;smithi181=j;smithi196=b;smithi196=e;smithi196=h;smithi196=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-05-03T20:36:11.409 INFO:teuthology.orchestra.run.smithi181.stdout:mgr 3/3 10s ago 8m smithi181=x;smithi196=y;smithi204=z;count:3 2024-05-03T20:36:11.409 INFO:teuthology.orchestra.run.smithi181.stdout:mon 3/3 10s ago 8m smithi181:172.21.15.181=a;smithi196:172.21.15.196=b;smithi204:172.21.15.204=c;count:3 2024-05-03T20:36:11.410 INFO:teuthology.orchestra.run.smithi181.stdout:osd 12 10s ago - 2024-05-03T20:36:11.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.412+0000 7f4ee85ef700 1 -- 172.21.15.181:0/3269078946 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4ecc0b2910 msgr2=0x7f4ecc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:11.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.412+0000 7f4ee85ef700 1 --2- 172.21.15.181:0/3269078946 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4ecc0b2910 0x7f4ecc0b4dd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4edc005fd0 tx=0x7f4edc005e50 comp rx=0 tx=0).stop 2024-05-03T20:36:11.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.412+0000 7f4ee85ef700 1 -- 172.21.15.181:0/3269078946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ee0105250 msgr2=0x7f4ee0102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:11.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.412+0000 7f4ee85ef700 1 --2- 172.21.15.181:0/3269078946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ee0105250 0x7f4ee0102da0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8007ee0 tx=0x7f4ed800d3b0 comp rx=0 tx=0).stop 2024-05-03T20:36:11.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.412+0000 7f4ee85ef700 1 -- 172.21.15.181:0/3269078946 shutdown_connections 2024-05-03T20:36:11.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.412+0000 7f4ee85ef700 1 --2- 172.21.15.181:0/3269078946 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f4ecc0b2910 0x7f4ecc0b4dd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:11.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.412+0000 7f4ee85ef700 1 --2- 172.21.15.181:0/3269078946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ee0105250 0x7f4ee0102da0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:11.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.412+0000 7f4ee85ef700 1 --2- 172.21.15.181:0/3269078946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ee0101830 0x7f4ee0104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:11.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.412+0000 7f4ee85ef700 1 --2- 172.21.15.181:0/3269078946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ee01048b0 0x7f4ee0102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:11.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.412+0000 7f4ee85ef700 1 -- 172.21.15.181:0/3269078946 >> 172.21.15.181:0/3269078946 conn(0x7f4ee00fd170 msgr2=0x7f4ee0107b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:11.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.412+0000 7f4ee85ef700 1 -- 172.21.15.181:0/3269078946 shutdown_connections 2024-05-03T20:36:11.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:11.412+0000 7f4ee85ef700 1 -- 172.21.15.181:0/3269078946 wait complete. 2024-05-03T20:36:11.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:11 smithi196 ceph-mon[32858]: pgmap v361: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:36:11.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:11 smithi204 ceph-mon[33595]: pgmap v361: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:36:11.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:11 smithi181 ceph-mon[29801]: pgmap v361: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:36:12.093 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-03T20:36:12.332 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:36:12.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:12 smithi204 ceph-mon[33595]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:36:12.704 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:12 smithi181 ceph-mon[29801]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:36:12.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:12 smithi196 ceph-mon[32858]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:36:13.447 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:13 smithi181 ceph-mon[29801]: pgmap v362: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:36:13.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:13 smithi204 ceph-mon[33595]: pgmap v362: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:36:13.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:13 smithi196 ceph-mon[32858]: pgmap v362: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:36:13.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.954+0000 7f04dd108700 1 -- 172.21.15.181:0/212456062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04d8103fa0 msgr2=0x7f04d810a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:13.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.954+0000 7f04dd108700 1 --2- 172.21.15.181:0/212456062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04d8103fa0 0x7f04d810a850 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f04d000b0d0 tx=0x7f04d000b3e0 comp rx=0 tx=0).stop 2024-05-03T20:36:13.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.954+0000 7f04dd108700 1 -- 172.21.15.181:0/212456062 shutdown_connections 2024-05-03T20:36:13.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.954+0000 7f04dd108700 1 --2- 172.21.15.181:0/212456062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04d8103fa0 0x7f04d810a850 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:13.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.954+0000 7f04dd108700 1 --2- 172.21.15.181:0/212456062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04d8102400 0x7f04d8102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:13.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.954+0000 7f04dd108700 1 --2- 172.21.15.181:0/212456062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04d8103600 0x7f04d8103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:13.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.954+0000 7f04dd108700 1 -- 172.21.15.181:0/212456062 >> 172.21.15.181:0/212456062 conn(0x7f04d80fd9a0 msgr2=0x7f04d80ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:13.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.954+0000 7f04dd108700 1 -- 172.21.15.181:0/212456062 shutdown_connections 2024-05-03T20:36:13.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.954+0000 7f04dd108700 1 -- 172.21.15.181:0/212456062 wait complete. 2024-05-03T20:36:13.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.955+0000 7f04dd108700 1 Processor -- start 2024-05-03T20:36:13.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.955+0000 7f04dd108700 1 -- start start 2024-05-03T20:36:13.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.955+0000 7f04dd108700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04d8102400 0x7f04d81996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:13.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.955+0000 7f04dd108700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04d8103600 0x7f04d8199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:13.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.955+0000 7f04dd108700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04d8103fa0 0x7f04d81a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:13.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.955+0000 7f04dd108700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f04d810df10 con 0x7f04d8102400 2024-05-03T20:36:13.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.955+0000 7f04dd108700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f04d810dd90 con 0x7f04d8103fa0 2024-05-03T20:36:13.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.955+0000 7f04dd108700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f04d810e090 con 0x7f04d8103600 2024-05-03T20:36:13.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.955+0000 7f04d6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04d8102400 0x7f04d81996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:13.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.955+0000 7f04d6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04d8102400 0x7f04d81996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42604/0 (socket says 172.21.15.181:42604) 2024-05-03T20:36:13.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.955+0000 7f04d6d9d700 1 -- 172.21.15.181:0/1923149294 learned_addr learned my addr 172.21.15.181:0/1923149294 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:13.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.955+0000 7f04d659c700 1 --2- 172.21.15.181:0/1923149294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04d8103600 0x7f04d8199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:13.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.955+0000 7f04d759e700 1 --2- 172.21.15.181:0/1923149294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04d8103fa0 0x7f04d81a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:13.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.955+0000 7f04d6d9d700 1 -- 172.21.15.181:0/1923149294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04d8103600 msgr2=0x7f04d8199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:13.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.955+0000 7f04d6d9d700 1 --2- 172.21.15.181:0/1923149294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04d8103600 0x7f04d8199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:13.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.955+0000 7f04d6d9d700 1 -- 172.21.15.181:0/1923149294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04d8103fa0 msgr2=0x7f04d81a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:13.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.955+0000 7f04d6d9d700 1 --2- 172.21.15.181:0/1923149294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04d8103fa0 0x7f04d81a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:13.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.955+0000 7f04d6d9d700 1 -- 172.21.15.181:0/1923149294 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d81a12c0 con 0x7f04d8102400 2024-05-03T20:36:13.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.955+0000 7f04d759e700 1 --2- 172.21.15.181:0/1923149294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04d8103fa0 0x7f04d81a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:36:13.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.956+0000 7f04d6d9d700 1 --2- 172.21.15.181:0/1923149294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04d8102400 0x7f04d81996b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f04c800b530 tx=0x7f04c800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:13.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.956+0000 7f04bffff700 1 -- 172.21.15.181:0/1923149294 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8015020 con 0x7f04d8102400 2024-05-03T20:36:13.958 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.956+0000 7f04bffff700 1 -- 172.21.15.181:0/1923149294 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04c8011ce0 con 0x7f04d8102400 2024-05-03T20:36:13.958 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.956+0000 7f04bffff700 1 -- 172.21.15.181:0/1923149294 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8010d60 con 0x7f04d8102400 2024-05-03T20:36:13.958 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.956+0000 7f04dd108700 1 -- 172.21.15.181:0/1923149294 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d81a15b0 con 0x7f04d8102400 2024-05-03T20:36:13.958 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.956+0000 7f04dd108700 1 -- 172.21.15.181:0/1923149294 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d81a1b70 con 0x7f04d8102400 2024-05-03T20:36:13.958 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.957+0000 7f04dd108700 1 -- 172.21.15.181:0/1923149294 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04d806a190 con 0x7f04d8102400 2024-05-03T20:36:13.959 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.958+0000 7f04bffff700 1 -- 172.21.15.181:0/1923149294 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f04c8018020 con 0x7f04d8102400 2024-05-03T20:36:13.959 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.959+0000 7f04bffff700 1 --2- 172.21.15.181:0/1923149294 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f04c00b2910 0x7f04c00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:13.959 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.959+0000 7f04d659c700 1 --2- 172.21.15.181:0/1923149294 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f04c00b2910 0x7f04c00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:13.959 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.959+0000 7f04bffff700 1 -- 172.21.15.181:0/1923149294 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f04c80e1fe0 con 0x7f04d8102400 2024-05-03T20:36:13.959 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.959+0000 7f04d659c700 1 --2- 172.21.15.181:0/1923149294 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f04c00b2910 0x7f04c00b4dd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f04cc005950 tx=0x7f04cc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:13.961 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:13.962+0000 7f04bffff700 1 -- 172.21.15.181:0/1923149294 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f04c801f020 con 0x7f04d8102400 2024-05-03T20:36:14.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:14.172+0000 7f04dd108700 1 -- 172.21.15.181:0/1923149294 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f04d80ffbb0 con 0x7f04c00b2910 2024-05-03T20:36:14.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:14.174+0000 7f04bffff700 1 -- 172.21.15.181:0/1923149294 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f04d80ffbb0 con 0x7f04c00b2910 2024-05-03T20:36:14.173 INFO:teuthology.orchestra.run.smithi181.stdout:HOST ADDR LABELS STATUS 2024-05-03T20:36:14.173 INFO:teuthology.orchestra.run.smithi181.stdout:smithi181 172.21.15.181 2024-05-03T20:36:14.173 INFO:teuthology.orchestra.run.smithi181.stdout:smithi196 172.21.15.196 2024-05-03T20:36:14.173 INFO:teuthology.orchestra.run.smithi181.stdout:smithi204 172.21.15.204 2024-05-03T20:36:14.173 INFO:teuthology.orchestra.run.smithi181.stdout:3 hosts in cluster 2024-05-03T20:36:14.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:14.176+0000 7f04dd108700 1 -- 172.21.15.181:0/1923149294 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f04c00b2910 msgr2=0x7f04c00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:14.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:14.176+0000 7f04dd108700 1 --2- 172.21.15.181:0/1923149294 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f04c00b2910 0x7f04c00b4dd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f04cc005950 tx=0x7f04cc00b410 comp rx=0 tx=0).stop 2024-05-03T20:36:14.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:14.176+0000 7f04dd108700 1 -- 172.21.15.181:0/1923149294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04d8102400 msgr2=0x7f04d81996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:14.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:14.176+0000 7f04dd108700 1 --2- 172.21.15.181:0/1923149294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04d8102400 0x7f04d81996b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f04c800b530 tx=0x7f04c800b840 comp rx=0 tx=0).stop 2024-05-03T20:36:14.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:14.176+0000 7f04dd108700 1 -- 172.21.15.181:0/1923149294 shutdown_connections 2024-05-03T20:36:14.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:14.176+0000 7f04dd108700 1 --2- 172.21.15.181:0/1923149294 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f04c00b2910 0x7f04c00b4dd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:14.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:14.176+0000 7f04dd108700 1 --2- 172.21.15.181:0/1923149294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04d8103fa0 0x7f04d81a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:14.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:14.176+0000 7f04dd108700 1 --2- 172.21.15.181:0/1923149294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04d8102400 0x7f04d81996b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:14.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:14.176+0000 7f04dd108700 1 --2- 172.21.15.181:0/1923149294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04d8103600 0x7f04d8199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:14.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:14.176+0000 7f04dd108700 1 -- 172.21.15.181:0/1923149294 >> 172.21.15.181:0/1923149294 conn(0x7f04d80fd9a0 msgr2=0x7f04d80ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:14.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:14.176+0000 7f04dd108700 1 -- 172.21.15.181:0/1923149294 shutdown_connections 2024-05-03T20:36:14.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:14.176+0000 7f04dd108700 1 -- 172.21.15.181:0/1923149294 wait complete. 2024-05-03T20:36:14.834 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-03T20:36:15.087 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:36:15.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:15 smithi181 ceph-mon[29801]: pgmap v363: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:36:15.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:15 smithi181 ceph-mon[29801]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:36:15.658 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:15 smithi196 ceph-mon[32858]: pgmap v363: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:36:15.658 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:15 smithi196 ceph-mon[32858]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:36:15.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:15 smithi204 ceph-mon[33595]: pgmap v363: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:36:15.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:15 smithi204 ceph-mon[33595]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:36:16.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.712+0000 7f812c60d700 1 -- 172.21.15.181:0/3730125829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8124103fc0 msgr2=0x7f812410a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:16.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.712+0000 7f812c60d700 1 --2- 172.21.15.181:0/3730125829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8124103fc0 0x7f812410a870 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f811c00b0d0 tx=0x7f811c00b3e0 comp rx=0 tx=0).stop 2024-05-03T20:36:16.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.712+0000 7f812c60d700 1 -- 172.21.15.181:0/3730125829 shutdown_connections 2024-05-03T20:36:16.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.712+0000 7f812c60d700 1 --2- 172.21.15.181:0/3730125829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8124103fc0 0x7f812410a870 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:16.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.712+0000 7f812c60d700 1 --2- 172.21.15.181:0/3730125829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8124102420 0x7f8124102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:16.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.712+0000 7f812c60d700 1 --2- 172.21.15.181:0/3730125829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8124103620 0x7f8124103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:16.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.712+0000 7f812c60d700 1 -- 172.21.15.181:0/3730125829 >> 172.21.15.181:0/3730125829 conn(0x7f81240fd9c0 msgr2=0x7f81240ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:16.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.712+0000 7f812c60d700 1 -- 172.21.15.181:0/3730125829 shutdown_connections 2024-05-03T20:36:16.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.712+0000 7f812c60d700 1 -- 172.21.15.181:0/3730125829 wait complete. 2024-05-03T20:36:16.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.713+0000 7f812c60d700 1 Processor -- start 2024-05-03T20:36:16.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.713+0000 7f812c60d700 1 -- start start 2024-05-03T20:36:16.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.713+0000 7f812c60d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8124102420 0x7f8124199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:16.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.713+0000 7f812c60d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8124103620 0x7f8124199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:16.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.713+0000 7f812c60d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8124103fc0 0x7f81241a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:16.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.713+0000 7f812c60d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f812410df30 con 0x7f8124103620 2024-05-03T20:36:16.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.713+0000 7f812c60d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f812410ddb0 con 0x7f8124102420 2024-05-03T20:36:16.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.713+0000 7f812c60d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f812410e0b0 con 0x7f8124103fc0 2024-05-03T20:36:16.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.713+0000 7f8129ba8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8124103620 0x7f8124199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:16.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.713+0000 7f8129ba8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8124103620 0x7f8124199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34060/0 (socket says 172.21.15.181:34060) 2024-05-03T20:36:16.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.713+0000 7f8129ba8700 1 -- 172.21.15.181:0/1776267757 learned_addr learned my addr 172.21.15.181:0/1776267757 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:16.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.713+0000 7f812a3a9700 1 --2- 172.21.15.181:0/1776267757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8124102420 0x7f8124199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:16.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.714+0000 7f8129ba8700 1 -- 172.21.15.181:0/1776267757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8124103fc0 msgr2=0x7f81241a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:16.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.714+0000 7f8129ba8700 1 --2- 172.21.15.181:0/1776267757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8124103fc0 0x7f81241a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:16.716 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.714+0000 7f8129ba8700 1 -- 172.21.15.181:0/1776267757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8124102420 msgr2=0x7f8124199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:16.716 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.714+0000 7f8129ba8700 1 --2- 172.21.15.181:0/1776267757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8124102420 0x7f8124199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:16.716 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.714+0000 7f8129ba8700 1 -- 172.21.15.181:0/1776267757 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81241a1260 con 0x7f8124103620 2024-05-03T20:36:16.716 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.714+0000 7f8129ba8700 1 --2- 172.21.15.181:0/1776267757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8124103620 0x7f8124199b40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f811400c950 tx=0x7f811400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:16.716 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.714+0000 7f811b7fe700 1 -- 172.21.15.181:0/1776267757 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8114007ac0 con 0x7f8124103620 2024-05-03T20:36:16.716 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.714+0000 7f812c60d700 1 -- 172.21.15.181:0/1776267757 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81241a1550 con 0x7f8124103620 2024-05-03T20:36:16.716 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.714+0000 7f811b7fe700 1 -- 172.21.15.181:0/1776267757 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8114004ce0 con 0x7f8124103620 2024-05-03T20:36:16.716 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.714+0000 7f811b7fe700 1 -- 172.21.15.181:0/1776267757 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8114005580 con 0x7f8124103620 2024-05-03T20:36:16.716 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.714+0000 7f812c60d700 1 -- 172.21.15.181:0/1776267757 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81241a1b10 con 0x7f8124103620 2024-05-03T20:36:16.716 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.715+0000 7f812c60d700 1 -- 172.21.15.181:0/1776267757 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f812406a190 con 0x7f8124103620 2024-05-03T20:36:16.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.716+0000 7f811b7fe700 1 -- 172.21.15.181:0/1776267757 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f8114007c60 con 0x7f8124103620 2024-05-03T20:36:16.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.717+0000 7f811b7fe700 1 --2- 172.21.15.181:0/1776267757 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f81100b2910 0x7f81100b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:16.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.717+0000 7f812a3a9700 1 --2- 172.21.15.181:0/1776267757 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f81100b2910 0x7f81100b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:16.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.717+0000 7f811b7fe700 1 -- 172.21.15.181:0/1776267757 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f81140e2350 con 0x7f8124103620 2024-05-03T20:36:16.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.717+0000 7f812a3a9700 1 --2- 172.21.15.181:0/1776267757 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f81100b2910 0x7f81100b4dd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8124103480 tx=0x7f8120009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:16.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.720+0000 7f811b7fe700 1 -- 172.21.15.181:0/1776267757 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8114014030 con 0x7f8124103620 2024-05-03T20:36:16.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.933+0000 7f812c60d700 1 -- 172.21.15.181:0/1776267757 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f81240ffbf0 con 0x7f81100b2910 2024-05-03T20:36:16.934 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.935+0000 7f811b7fe700 1 -- 172.21.15.181:0/1776267757 <== mgr.14152 v2:172.21.15.181:6800/1919423525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f81240ffbf0 con 0x7f81100b2910 2024-05-03T20:36:16.935 INFO:teuthology.orchestra.run.smithi181.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-03T20:36:16.935 INFO:teuthology.orchestra.run.smithi181.stdout:smithi181 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6246000E400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-03T20:36:16.935 INFO:teuthology.orchestra.run.smithi181.stdout:smithi196 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6246000Z400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-03T20:36:16.935 INFO:teuthology.orchestra.run.smithi181.stdout:smithi204 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GD400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-05-03T20:36:16.937 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.937+0000 7f812c60d700 1 -- 172.21.15.181:0/1776267757 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f81100b2910 msgr2=0x7f81100b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:16.937 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.937+0000 7f812c60d700 1 --2- 172.21.15.181:0/1776267757 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f81100b2910 0x7f81100b4dd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8124103480 tx=0x7f8120009190 comp rx=0 tx=0).stop 2024-05-03T20:36:16.937 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.937+0000 7f812c60d700 1 -- 172.21.15.181:0/1776267757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8124103620 msgr2=0x7f8124199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:16.937 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.937+0000 7f812c60d700 1 --2- 172.21.15.181:0/1776267757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8124103620 0x7f8124199b40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f811400c950 tx=0x7f811400cc60 comp rx=0 tx=0).stop 2024-05-03T20:36:16.937 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.937+0000 7f812c60d700 1 -- 172.21.15.181:0/1776267757 shutdown_connections 2024-05-03T20:36:16.937 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.937+0000 7f812c60d700 1 --2- 172.21.15.181:0/1776267757 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f81100b2910 0x7f81100b4dd0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:16.937 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.937+0000 7f812c60d700 1 --2- 172.21.15.181:0/1776267757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8124103fc0 0x7f81241a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:16.937 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.937+0000 7f812c60d700 1 --2- 172.21.15.181:0/1776267757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8124102420 0x7f8124199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:16.938 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.937+0000 7f812c60d700 1 --2- 172.21.15.181:0/1776267757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8124103620 0x7f8124199b40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:16.938 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.937+0000 7f812c60d700 1 -- 172.21.15.181:0/1776267757 >> 172.21.15.181:0/1776267757 conn(0x7f81240fd9c0 msgr2=0x7f81240ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:16.938 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.937+0000 7f812c60d700 1 -- 172.21.15.181:0/1776267757 shutdown_connections 2024-05-03T20:36:16.938 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:16.937+0000 7f812c60d700 1 -- 172.21.15.181:0/1776267757 wait complete. 2024-05-03T20:36:17.618 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:17 smithi181 ceph-mon[29801]: pgmap v364: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:36:17.619 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-03T20:36:17.630 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi181.front.sepia.ceph.com 2024-05-03T20:36:17.630 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-03T20:36:17.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:17 smithi196 ceph-mon[32858]: pgmap v364: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:36:17.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:17 smithi204 ceph-mon[33595]: pgmap v364: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:36:17.879 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:36:18.694 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:18 smithi181 ceph-mon[29801]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:36:18.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:18 smithi196 ceph-mon[32858]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:36:18.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:18 smithi204 ceph-mon[33595]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:36:19.526 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.524+0000 7f8c04667700 1 -- 172.21.15.181:0/2339785161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bfc103600 msgr2=0x7f8bfc103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:19.526 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.524+0000 7f8c04667700 1 --2- 172.21.15.181:0/2339785161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bfc103600 0x7f8bfc103a60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8bec0099c0 tx=0x7f8bec009cd0 comp rx=0 tx=0).stop 2024-05-03T20:36:19.526 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.524+0000 7f8c04667700 1 -- 172.21.15.181:0/2339785161 shutdown_connections 2024-05-03T20:36:19.526 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.524+0000 7f8c04667700 1 --2- 172.21.15.181:0/2339785161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bfc103fa0 0x7f8bfc10a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:19.526 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.524+0000 7f8c04667700 1 --2- 172.21.15.181:0/2339785161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bfc102400 0x7f8bfc102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:19.526 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.524+0000 7f8c04667700 1 --2- 172.21.15.181:0/2339785161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bfc103600 0x7f8bfc103a60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:19.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.524+0000 7f8c04667700 1 -- 172.21.15.181:0/2339785161 >> 172.21.15.181:0/2339785161 conn(0x7f8bfc0fd9a0 msgr2=0x7f8bfc0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:19.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.525+0000 7f8c04667700 1 -- 172.21.15.181:0/2339785161 shutdown_connections 2024-05-03T20:36:19.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.525+0000 7f8c04667700 1 -- 172.21.15.181:0/2339785161 wait complete. 2024-05-03T20:36:19.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.525+0000 7f8c04667700 1 Processor -- start 2024-05-03T20:36:19.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.525+0000 7f8c04667700 1 -- start start 2024-05-03T20:36:19.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.525+0000 7f8c04667700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bfc102400 0x7f8bfc110410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:19.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.525+0000 7f8c04667700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bfc103600 0x7f8bfc10ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:19.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.525+0000 7f8c04667700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bfc103fa0 0x7f8bfc10efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:19.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.526+0000 7f8c04667700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc10ded0 con 0x7f8bfc103fa0 2024-05-03T20:36:19.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.526+0000 7f8c04667700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc10dd50 con 0x7f8bfc103600 2024-05-03T20:36:19.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.526+0000 7f8c04667700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc10e050 con 0x7f8bfc102400 2024-05-03T20:36:19.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.526+0000 7f8c02c04700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bfc103fa0 0x7f8bfc10efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:19.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.526+0000 7f8c02c04700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bfc103fa0 0x7f8bfc10efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34080/0 (socket says 172.21.15.181:34080) 2024-05-03T20:36:19.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.526+0000 7f8c02c04700 1 -- 172.21.15.181:0/4091420769 learned_addr learned my addr 172.21.15.181:0/4091420769 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:19.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.526+0000 7f8c01c02700 1 --2- 172.21.15.181:0/4091420769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bfc103600 0x7f8bfc10ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:19.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.526+0000 7f8c02403700 1 --2- 172.21.15.181:0/4091420769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bfc102400 0x7f8bfc110410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:19.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.526+0000 7f8c02c04700 1 -- 172.21.15.181:0/4091420769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bfc102400 msgr2=0x7f8bfc110410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:19.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.526+0000 7f8c02c04700 1 --2- 172.21.15.181:0/4091420769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bfc102400 0x7f8bfc110410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:19.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.526+0000 7f8c02c04700 1 -- 172.21.15.181:0/4091420769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bfc103600 msgr2=0x7f8bfc10ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:19.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.526+0000 7f8c02c04700 1 --2- 172.21.15.181:0/4091420769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bfc103600 0x7f8bfc10ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:19.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.526+0000 7f8c02c04700 1 -- 172.21.15.181:0/4091420769 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bfc10f850 con 0x7f8bfc103fa0 2024-05-03T20:36:19.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.526+0000 7f8c02403700 1 --2- 172.21.15.181:0/4091420769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bfc102400 0x7f8bfc110410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:36:19.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.526+0000 7f8c02c04700 1 --2- 172.21.15.181:0/4091420769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bfc103fa0 0x7f8bfc10efa0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400ada0 tx=0x7f8bf400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:19.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.526+0000 7f8bf37fe700 1 -- 172.21.15.181:0/4091420769 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4017020 con 0x7f8bfc103fa0 2024-05-03T20:36:19.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.526+0000 7f8c04667700 1 -- 172.21.15.181:0/4091420769 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bfc1a37a0 con 0x7f8bfc103fa0 2024-05-03T20:36:19.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.526+0000 7f8bf37fe700 1 -- 172.21.15.181:0/4091420769 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bf400ce30 con 0x7f8bfc103fa0 2024-05-03T20:36:19.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.526+0000 7f8bf37fe700 1 -- 172.21.15.181:0/4091420769 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf40054a0 con 0x7f8bfc103fa0 2024-05-03T20:36:19.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.526+0000 7f8c04667700 1 -- 172.21.15.181:0/4091420769 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bfc1a3cb0 con 0x7f8bfc103fa0 2024-05-03T20:36:19.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.527+0000 7f8c04667700 1 -- 172.21.15.181:0/4091420769 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bfc06a190 con 0x7f8bfc103fa0 2024-05-03T20:36:19.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.529+0000 7f8bf37fe700 1 -- 172.21.15.181:0/4091420769 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f8bf400c980 con 0x7f8bfc103fa0 2024-05-03T20:36:19.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.529+0000 7f8bf37fe700 1 --2- 172.21.15.181:0/4091420769 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8be80b2910 0x7f8be80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:19.530 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.529+0000 7f8c02403700 1 --2- 172.21.15.181:0/4091420769 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8be80b2910 0x7f8be80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:19.530 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.529+0000 7f8bf37fe700 1 -- 172.21.15.181:0/4091420769 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f8bf40e1b90 con 0x7f8bfc103fa0 2024-05-03T20:36:19.530 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.530+0000 7f8c02403700 1 --2- 172.21.15.181:0/4091420769 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8be80b2910 0x7f8be80b4dd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8005fd0 tx=0x7f8bf8005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:19.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.534+0000 7f8bf37fe700 1 -- 172.21.15.181:0/4091420769 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8bf4021020 con 0x7f8bfc103fa0 2024-05-03T20:36:19.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:19 smithi181 ceph-mon[29801]: pgmap v365: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:36:19.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.783+0000 7f8c04667700 1 -- 172.21.15.181:0/4091420769 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f8bfc1a4030 con 0x7f8bfc103fa0 2024-05-03T20:36:19.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.784+0000 7f8bf37fe700 1 -- 172.21.15.181:0/4091420769 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3106 (secure 0 0 0) 0x7f8bf40a97e0 con 0x7f8bfc103fa0 2024-05-03T20:36:19.783 INFO:teuthology.orchestra.run.smithi181.stdout:e27 2024-05-03T20:36:19.783 INFO:teuthology.orchestra.run.smithi181.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-03T20:36:19.783 INFO:teuthology.orchestra.run.smithi181.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-03T20:36:19.784 INFO:teuthology.orchestra.run.smithi181.stdout:legacy client fscid: 1 2024-05-03T20:36:19.784 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:36:19.784 INFO:teuthology.orchestra.run.smithi181.stdout:Filesystem 'cephfs' (1) 2024-05-03T20:36:19.784 INFO:teuthology.orchestra.run.smithi181.stdout:fs_name cephfs 2024-05-03T20:36:19.784 INFO:teuthology.orchestra.run.smithi181.stdout:epoch 27 2024-05-03T20:36:19.784 INFO:teuthology.orchestra.run.smithi181.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-03T20:36:19.784 INFO:teuthology.orchestra.run.smithi181.stdout:created 2024-05-03T20:33:42.487965+0000 2024-05-03T20:36:19.784 INFO:teuthology.orchestra.run.smithi181.stdout:modified 2024-05-03T20:34:00.359735+0000 2024-05-03T20:36:19.784 INFO:teuthology.orchestra.run.smithi181.stdout:tableserver 0 2024-05-03T20:36:19.784 INFO:teuthology.orchestra.run.smithi181.stdout:root 0 2024-05-03T20:36:19.784 INFO:teuthology.orchestra.run.smithi181.stdout:session_timeout 300 2024-05-03T20:36:19.784 INFO:teuthology.orchestra.run.smithi181.stdout:session_autoclose 300 2024-05-03T20:36:19.784 INFO:teuthology.orchestra.run.smithi181.stdout:max_file_size 1099511627776 2024-05-03T20:36:19.784 INFO:teuthology.orchestra.run.smithi181.stdout:max_xattr_size 65536 2024-05-03T20:36:19.784 INFO:teuthology.orchestra.run.smithi181.stdout:required_client_features {} 2024-05-03T20:36:19.784 INFO:teuthology.orchestra.run.smithi181.stdout:last_failure 0 2024-05-03T20:36:19.785 INFO:teuthology.orchestra.run.smithi181.stdout:last_failure_osd_epoch 0 2024-05-03T20:36:19.785 INFO:teuthology.orchestra.run.smithi181.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-03T20:36:19.785 INFO:teuthology.orchestra.run.smithi181.stdout:max_mds 5 2024-05-03T20:36:19.785 INFO:teuthology.orchestra.run.smithi181.stdout:in 0,1,2,3,4 2024-05-03T20:36:19.785 INFO:teuthology.orchestra.run.smithi181.stdout:up {0=24488,1=24475,2=24464,3=24463,4=24457} 2024-05-03T20:36:19.785 INFO:teuthology.orchestra.run.smithi181.stdout:failed 2024-05-03T20:36:19.785 INFO:teuthology.orchestra.run.smithi181.stdout:damaged 2024-05-03T20:36:19.785 INFO:teuthology.orchestra.run.smithi181.stdout:stopped 2024-05-03T20:36:19.785 INFO:teuthology.orchestra.run.smithi181.stdout:data_pools [3] 2024-05-03T20:36:19.785 INFO:teuthology.orchestra.run.smithi181.stdout:metadata_pool 2 2024-05-03T20:36:19.785 INFO:teuthology.orchestra.run.smithi181.stdout:inline_data disabled 2024-05-03T20:36:19.785 INFO:teuthology.orchestra.run.smithi181.stdout:balancer 2024-05-03T20:36:19.785 INFO:teuthology.orchestra.run.smithi181.stdout:bal_rank_mask -1 2024-05-03T20:36:19.785 INFO:teuthology.orchestra.run.smithi181.stdout:standby_count_wanted 0 2024-05-03T20:36:19.785 INFO:teuthology.orchestra.run.smithi181.stdout:qdb_cluster leader: 24488 members: 24457,24464,24463,24475,24488 2024-05-03T20:36:19.785 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.b{0:24488} state up:active seq 5 addr [v2:172.21.15.196:6838/1182826743,v1:172.21.15.196:6839/1182826743] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T20:36:19.786 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.e{0:24437} state up:standby-replay seq 1 addr [v2:172.21.15.196:6834/1820206106,v1:172.21.15.196:6835/1820206106] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T20:36:19.786 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.i{1:24475} state up:active seq 11 addr [v2:172.21.15.204:6838/1644828559,v1:172.21.15.204:6839/1644828559] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T20:36:19.786 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.l{1:24433} state up:standby-replay seq 1 addr [v2:172.21.15.204:6832/2409628823,v1:172.21.15.204:6833/2409628823] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T20:36:19.786 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.h{2:24464} state up:active seq 11 addr [v2:172.21.15.196:6836/1529420259,v1:172.21.15.196:6837/1529420259] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T20:36:19.786 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.k{2:24410} state up:standby-replay seq 1 addr [v2:172.21.15.196:6832/3975688022,v1:172.21.15.196:6833/3975688022] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T20:36:19.786 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.c{3:24463} state up:active seq 12 addr [v2:172.21.15.204:6836/3006728559,v1:172.21.15.204:6837/3006728559] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T20:36:19.786 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.a{3:14574} state up:standby-replay seq 1 addr [v2:172.21.15.181:6840/2403160115,v1:172.21.15.181:6841/2403160115] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T20:36:19.786 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.f{4:24457} state up:active seq 13 addr [v2:172.21.15.204:6834/3870850287,v1:172.21.15.204:6835/3870850287] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T20:36:19.786 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.g{4:14562} state up:standby-replay seq 1 addr [v2:172.21.15.181:6838/257361276,v1:172.21.15.181:6839/257361276] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T20:36:19.786 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:36:19.786 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:36:19.786 INFO:teuthology.orchestra.run.smithi181.stdout:Standby daemons: 2024-05-03T20:36:19.786 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:36:19.786 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.j{-1:14508} state up:standby seq 1 addr [v2:172.21.15.181:6834/2414172454,v1:172.21.15.181:6835/2414172454] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T20:36:19.786 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.d{-1:14520} state up:standby seq 1 addr [v2:172.21.15.181:6836/204127189,v1:172.21.15.181:6837/204127189] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T20:36:19.787 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.786+0000 7f8c04667700 1 -- 172.21.15.181:0/4091420769 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8be80b2910 msgr2=0x7f8be80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:19.787 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.786+0000 7f8c04667700 1 --2- 172.21.15.181:0/4091420769 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8be80b2910 0x7f8be80b4dd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8005fd0 tx=0x7f8bf8005e40 comp rx=0 tx=0).stop 2024-05-03T20:36:19.788 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.786+0000 7f8c04667700 1 -- 172.21.15.181:0/4091420769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bfc103fa0 msgr2=0x7f8bfc10efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:19.788 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.786+0000 7f8c04667700 1 --2- 172.21.15.181:0/4091420769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bfc103fa0 0x7f8bfc10efa0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400ada0 tx=0x7f8bf400c330 comp rx=0 tx=0).stop 2024-05-03T20:36:19.788 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.786+0000 7f8c04667700 1 -- 172.21.15.181:0/4091420769 shutdown_connections 2024-05-03T20:36:19.788 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.786+0000 7f8c04667700 1 --2- 172.21.15.181:0/4091420769 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8be80b2910 0x7f8be80b4dd0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:19.788 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.786+0000 7f8c04667700 1 --2- 172.21.15.181:0/4091420769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bfc103fa0 0x7f8bfc10efa0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:19.788 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.786+0000 7f8c04667700 1 --2- 172.21.15.181:0/4091420769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bfc102400 0x7f8bfc110410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:19.788 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.786+0000 7f8c04667700 1 --2- 172.21.15.181:0/4091420769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bfc103600 0x7f8bfc10ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:19.788 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.786+0000 7f8c04667700 1 -- 172.21.15.181:0/4091420769 >> 172.21.15.181:0/4091420769 conn(0x7f8bfc0fd9a0 msgr2=0x7f8bfc0fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:19.788 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.786+0000 7f8c04667700 1 -- 172.21.15.181:0/4091420769 shutdown_connections 2024-05-03T20:36:19.788 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:19.786+0000 7f8c04667700 1 -- 172.21.15.181:0/4091420769 wait complete. 2024-05-03T20:36:19.788 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 27 2024-05-03T20:36:19.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:19 smithi196 ceph-mon[32858]: pgmap v365: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:36:19.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:19 smithi204 ceph-mon[33595]: pgmap v365: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:36:20.427 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-03T20:36:20.667 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:36:20.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:20 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/4091420769' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-03T20:36:20.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:20 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/4091420769' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-03T20:36:20.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:20 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/4091420769' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-03T20:36:21.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:21 smithi181 ceph-mon[29801]: pgmap v366: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:36:21.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:21 smithi196 ceph-mon[32858]: pgmap v366: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:36:21.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:21 smithi204 ceph-mon[33595]: pgmap v366: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:36:22.332 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.332+0000 7fb914eef700 1 -- 172.21.15.181:0/652031555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9101091f0 msgr2=0x7fb91010b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:22.332 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.332+0000 7fb914eef700 1 --2- 172.21.15.181:0/652031555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9101091f0 0x7fb91010b660 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb90800b0a0 tx=0x7fb90800b3b0 comp rx=0 tx=0).stop 2024-05-03T20:36:22.333 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.332+0000 7fb914eef700 1 -- 172.21.15.181:0/652031555 shutdown_connections 2024-05-03T20:36:22.333 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.332+0000 7fb914eef700 1 --2- 172.21.15.181:0/652031555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9101091f0 0x7fb91010b660 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:22.333 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.332+0000 7fb914eef700 1 --2- 172.21.15.181:0/652031555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb910103f90 0x7fb910106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:22.333 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.332+0000 7fb914eef700 1 --2- 172.21.15.181:0/652031555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9101068c0 0x7fb910108cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:22.333 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.332+0000 7fb914eef700 1 -- 172.21.15.181:0/652031555 >> 172.21.15.181:0/652031555 conn(0x7fb9100fd9a0 msgr2=0x7fb9100ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:22.333 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.333+0000 7fb914eef700 1 -- 172.21.15.181:0/652031555 shutdown_connections 2024-05-03T20:36:22.333 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.333+0000 7fb914eef700 1 -- 172.21.15.181:0/652031555 wait complete. 2024-05-03T20:36:22.335 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.333+0000 7fb914eef700 1 Processor -- start 2024-05-03T20:36:22.335 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.334+0000 7fb914eef700 1 -- start start 2024-05-03T20:36:22.335 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.334+0000 7fb914eef700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb910103f90 0x7fb910199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:22.335 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.334+0000 7fb914eef700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9101068c0 0x7fb910199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:22.335 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.334+0000 7fb914eef700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9101091f0 0x7fb9101a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:22.335 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.334+0000 7fb914eef700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb91010f060 con 0x7fb910103f90 2024-05-03T20:36:22.335 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.334+0000 7fb914eef700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb91010eee0 con 0x7fb9101091f0 2024-05-03T20:36:22.335 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.334+0000 7fb914eef700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb91010f1e0 con 0x7fb9101068c0 2024-05-03T20:36:22.335 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.334+0000 7fb90e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb910103f90 0x7fb910199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:22.335 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.334+0000 7fb90e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb910103f90 0x7fb910199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34106/0 (socket says 172.21.15.181:34106) 2024-05-03T20:36:22.335 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.334+0000 7fb90e59c700 1 -- 172.21.15.181:0/1328495702 learned_addr learned my addr 172.21.15.181:0/1328495702 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:22.335 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.334+0000 7fb905d9b700 1 --2- 172.21.15.181:0/1328495702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9101068c0 0x7fb910199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:22.336 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.334+0000 7fb90ed9d700 1 --2- 172.21.15.181:0/1328495702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9101091f0 0x7fb9101a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:22.336 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.334+0000 7fb90e59c700 1 -- 172.21.15.181:0/1328495702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9101068c0 msgr2=0x7fb910199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:22.336 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.334+0000 7fb90e59c700 1 --2- 172.21.15.181:0/1328495702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9101068c0 0x7fb910199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:22.336 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.334+0000 7fb90e59c700 1 -- 172.21.15.181:0/1328495702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9101091f0 msgr2=0x7fb9101a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:22.336 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.334+0000 7fb90e59c700 1 --2- 172.21.15.181:0/1328495702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9101091f0 0x7fb9101a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:22.336 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.334+0000 7fb90e59c700 1 -- 172.21.15.181:0/1328495702 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9101a13c0 con 0x7fb910103f90 2024-05-03T20:36:22.336 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.335+0000 7fb90e59c700 1 --2- 172.21.15.181:0/1328495702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb910103f90 0x7fb910199760 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb90000b560 tx=0x7fb90000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:22.336 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.335+0000 7fb907fff700 1 -- 172.21.15.181:0/1328495702 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb900015020 con 0x7fb910103f90 2024-05-03T20:36:22.336 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.335+0000 7fb914eef700 1 -- 172.21.15.181:0/1328495702 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9101a16b0 con 0x7fb910103f90 2024-05-03T20:36:22.336 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.335+0000 7fb907fff700 1 -- 172.21.15.181:0/1328495702 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb900011ce0 con 0x7fb910103f90 2024-05-03T20:36:22.336 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.335+0000 7fb907fff700 1 -- 172.21.15.181:0/1328495702 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb900010d40 con 0x7fb910103f90 2024-05-03T20:36:22.336 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.335+0000 7fb914eef700 1 -- 172.21.15.181:0/1328495702 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9101a1c70 con 0x7fb910103f90 2024-05-03T20:36:22.337 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.336+0000 7fb914eef700 1 -- 172.21.15.181:0/1328495702 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb91006a190 con 0x7fb910103f90 2024-05-03T20:36:22.337 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.338+0000 7fb907fff700 1 -- 172.21.15.181:0/1328495702 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fb900018020 con 0x7fb910103f90 2024-05-03T20:36:22.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.339+0000 7fb907fff700 1 --2- 172.21.15.181:0/1328495702 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb8fc0b2910 0x7fb8fc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:22.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.339+0000 7fb905d9b700 1 --2- 172.21.15.181:0/1328495702 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb8fc0b2910 0x7fb8fc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:22.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.339+0000 7fb907fff700 1 -- 172.21.15.181:0/1328495702 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7fb9000e1ec0 con 0x7fb910103f90 2024-05-03T20:36:22.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.339+0000 7fb905d9b700 1 --2- 172.21.15.181:0/1328495702 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb8fc0b2910 0x7fb8fc0b4dd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8009a10 tx=0x7fb8f8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:22.341 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.342+0000 7fb907fff700 1 -- 172.21.15.181:0/1328495702 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb90001f020 con 0x7fb910103f90 2024-05-03T20:36:22.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.548+0000 7fb914eef700 1 -- 172.21.15.181:0/1328495702 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fb91002e590 con 0x7fb910103f90 2024-05-03T20:36:22.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.549+0000 7fb907fff700 1 -- 172.21.15.181:0/1328495702 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v83) ==== 56+0+5305 (secure 0 0 0) 0x7fb9000a9b90 con 0x7fb910103f90 2024-05-03T20:36:22.548 INFO:teuthology.orchestra.run.smithi181.stdout:epoch 83 2024-05-03T20:36:22.548 INFO:teuthology.orchestra.run.smithi181.stdout:fsid 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:36:22.549 INFO:teuthology.orchestra.run.smithi181.stdout:created 2024-05-03T20:25:39.427815+0000 2024-05-03T20:36:22.549 INFO:teuthology.orchestra.run.smithi181.stdout:modified 2024-05-03T20:34:35.581238+0000 2024-05-03T20:36:22.549 INFO:teuthology.orchestra.run.smithi181.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-03T20:36:22.549 INFO:teuthology.orchestra.run.smithi181.stdout:crush_version 26 2024-05-03T20:36:22.549 INFO:teuthology.orchestra.run.smithi181.stdout:full_ratio 0.95 2024-05-03T20:36:22.549 INFO:teuthology.orchestra.run.smithi181.stdout:backfillfull_ratio 0.9 2024-05-03T20:36:22.549 INFO:teuthology.orchestra.run.smithi181.stdout:nearfull_ratio 0.85 2024-05-03T20:36:22.549 INFO:teuthology.orchestra.run.smithi181.stdout:require_min_compat_client luminous 2024-05-03T20:36:22.549 INFO:teuthology.orchestra.run.smithi181.stdout:min_compat_client luminous 2024-05-03T20:36:22.549 INFO:teuthology.orchestra.run.smithi181.stdout:require_osd_release squid 2024-05-03T20:36:22.549 INFO:teuthology.orchestra.run.smithi181.stdout:stretch_mode_enabled false 2024-05-03T20:36:22.549 INFO:teuthology.orchestra.run.smithi181.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-03T20:36:22.549 INFO:teuthology.orchestra.run.smithi181.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-03T20:36:22.549 INFO:teuthology.orchestra.run.smithi181.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-03T20:36:22.549 INFO:teuthology.orchestra.run.smithi181.stdout:max_osd 12 2024-05-03T20:36:22.550 INFO:teuthology.orchestra.run.smithi181.stdout:osd.0 up in weight 1 up_from 8 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6802/1683973278,v1:172.21.15.181:6803/1683973278] [v2:172.21.15.181:6804/1683973278,v1:172.21.15.181:6805/1683973278] exists,up 5f594d97-10a4-4f77-aebc-c00dbe92a4af 2024-05-03T20:36:22.550 INFO:teuthology.orchestra.run.smithi181.stdout:osd.1 up in weight 1 up_from 13 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6800/2136873375,v1:172.21.15.196:6801/2136873375] [v2:172.21.15.196:6802/2136873375,v1:172.21.15.196:6803/2136873375] exists,up 9726d641-8930-4b45-9192-6ee2b6bfcdcc 2024-05-03T20:36:22.550 INFO:teuthology.orchestra.run.smithi181.stdout:osd.2 up in weight 1 up_from 18 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6800/3291673137,v1:172.21.15.204:6801/3291673137] [v2:172.21.15.204:6802/3291673137,v1:172.21.15.204:6803/3291673137] exists,up 3d5573cc-37c1-4c3b-a61e-2363ef3ff964 2024-05-03T20:36:22.550 INFO:teuthology.orchestra.run.smithi181.stdout:osd.3 up in weight 1 up_from 27 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6810/29064971,v1:172.21.15.181:6811/29064971] [v2:172.21.15.181:6812/29064971,v1:172.21.15.181:6813/29064971] exists,up b9b205e3-a335-4771-9a66-55cfe6a8c31a 2024-05-03T20:36:22.550 INFO:teuthology.orchestra.run.smithi181.stdout:osd.4 up in weight 1 up_from 33 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6808/1145408995,v1:172.21.15.196:6809/1145408995] [v2:172.21.15.196:6810/1145408995,v1:172.21.15.196:6811/1145408995] exists,up 550f01ef-1947-4f9f-b759-34e3ea831839 2024-05-03T20:36:22.550 INFO:teuthology.orchestra.run.smithi181.stdout:osd.5 up in weight 1 up_from 38 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6808/855680559,v1:172.21.15.204:6809/855680559] [v2:172.21.15.204:6810/855680559,v1:172.21.15.204:6811/855680559] exists,up 6b74cb3a-f39a-4131-8793-1d5286856aa6 2024-05-03T20:36:22.550 INFO:teuthology.orchestra.run.smithi181.stdout:osd.6 up in weight 1 up_from 44 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6818/1561969338,v1:172.21.15.181:6819/1561969338] [v2:172.21.15.181:6820/1561969338,v1:172.21.15.181:6821/1561969338] exists,up dbe19bdb-08eb-40aa-842b-0710963d4497 2024-05-03T20:36:22.550 INFO:teuthology.orchestra.run.smithi181.stdout:osd.7 up in weight 1 up_from 49 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6816/824890163,v1:172.21.15.196:6817/824890163] [v2:172.21.15.196:6818/824890163,v1:172.21.15.196:6819/824890163] exists,up 016c145c-ad67-42ac-9775-ca2840790abc 2024-05-03T20:36:22.550 INFO:teuthology.orchestra.run.smithi181.stdout:osd.8 up in weight 1 up_from 56 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6816/3566909552,v1:172.21.15.204:6817/3566909552] [v2:172.21.15.204:6818/3566909552,v1:172.21.15.204:6819/3566909552] exists,up 10235638-c1f1-41d3-8e24-f8b2344bfeff 2024-05-03T20:36:22.550 INFO:teuthology.orchestra.run.smithi181.stdout:osd.9 up in weight 1 up_from 62 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6826/241693472,v1:172.21.15.181:6827/241693472] [v2:172.21.15.181:6828/241693472,v1:172.21.15.181:6829/241693472] exists,up 96ce5aa6-e6a9-4d83-8562-589ac94c8cd1 2024-05-03T20:36:22.550 INFO:teuthology.orchestra.run.smithi181.stdout:osd.10 up in weight 1 up_from 67 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6824/3980645995,v1:172.21.15.196:6825/3980645995] [v2:172.21.15.196:6826/3980645995,v1:172.21.15.196:6827/3980645995] exists,up d6c9ffab-731c-46d5-88cd-5f21ba5acfa0 2024-05-03T20:36:22.550 INFO:teuthology.orchestra.run.smithi181.stdout:osd.11 up in weight 1 up_from 73 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6824/488710574,v1:172.21.15.204:6825/488710574] [v2:172.21.15.204:6826/488710574,v1:172.21.15.204:6827/488710574] exists,up 3f58a2d6-bbbb-4e4d-bf83-2c886dd9484b 2024-05-03T20:36:22.550 INFO:teuthology.orchestra.run.smithi181.stdout:pg_upmap_items 2.0 [8,4] 2024-05-03T20:36:22.551 INFO:teuthology.orchestra.run.smithi181.stdout:pg_upmap_items 2.c [2,4] 2024-05-03T20:36:22.551 INFO:teuthology.orchestra.run.smithi181.stdout:pg_upmap_items 2.16 [2,6,7,11] 2024-05-03T20:36:22.551 INFO:teuthology.orchestra.run.smithi181.stdout:pg_upmap_items 2.1d [8,11] 2024-05-03T20:36:22.551 INFO:teuthology.orchestra.run.smithi181.stdout:pg_upmap_items 3.1b [4,8,5,9] 2024-05-03T20:36:22.551 INFO:teuthology.orchestra.run.smithi181.stdout:pg_upmap_items 3.21 [4,6] 2024-05-03T20:36:22.551 INFO:teuthology.orchestra.run.smithi181.stdout:pg_upmap_items 3.29 [3,9] 2024-05-03T20:36:22.551 INFO:teuthology.orchestra.run.smithi181.stdout:pg_upmap_items 3.36 [4,9] 2024-05-03T20:36:22.551 INFO:teuthology.orchestra.run.smithi181.stdout:blocklist 172.21.15.181:6800/2389231557 expires 2024-05-04T20:26:33.766772+0000 2024-05-03T20:36:22.551 INFO:teuthology.orchestra.run.smithi181.stdout:blocklist 172.21.15.181:0/3398276284 expires 2024-05-04T20:26:06.561713+0000 2024-05-03T20:36:22.551 INFO:teuthology.orchestra.run.smithi181.stdout:blocklist 172.21.15.181:6801/2389231557 expires 2024-05-04T20:26:33.766772+0000 2024-05-03T20:36:22.551 INFO:teuthology.orchestra.run.smithi181.stdout:blocklist 172.21.15.181:0/2327733609 expires 2024-05-04T20:26:06.561713+0000 2024-05-03T20:36:22.551 INFO:teuthology.orchestra.run.smithi181.stdout:blocklist 172.21.15.181:0/628540759 expires 2024-05-04T20:26:33.766772+0000 2024-05-03T20:36:22.551 INFO:teuthology.orchestra.run.smithi181.stdout:blocklist 172.21.15.181:0/3387163329 expires 2024-05-04T20:26:06.561713+0000 2024-05-03T20:36:22.551 INFO:teuthology.orchestra.run.smithi181.stdout:blocklist 172.21.15.181:6800/1802984663 expires 2024-05-04T20:26:06.561713+0000 2024-05-03T20:36:22.551 INFO:teuthology.orchestra.run.smithi181.stdout:blocklist 172.21.15.181:0/67924109 expires 2024-05-04T20:26:33.766772+0000 2024-05-03T20:36:22.552 INFO:teuthology.orchestra.run.smithi181.stdout:blocklist 172.21.15.181:0/3693389657 expires 2024-05-04T20:26:33.766772+0000 2024-05-03T20:36:22.552 INFO:teuthology.orchestra.run.smithi181.stdout:blocklist 172.21.15.181:6801/1802984663 expires 2024-05-04T20:26:06.561713+0000 2024-05-03T20:36:22.552 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.551+0000 7fb914eef700 1 -- 172.21.15.181:0/1328495702 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb8fc0b2910 msgr2=0x7fb8fc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:22.552 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.551+0000 7fb914eef700 1 --2- 172.21.15.181:0/1328495702 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb8fc0b2910 0x7fb8fc0b4dd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8009a10 tx=0x7fb8f8009190 comp rx=0 tx=0).stop 2024-05-03T20:36:22.552 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.551+0000 7fb914eef700 1 -- 172.21.15.181:0/1328495702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb910103f90 msgr2=0x7fb910199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:22.552 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.551+0000 7fb914eef700 1 --2- 172.21.15.181:0/1328495702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb910103f90 0x7fb910199760 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb90000b560 tx=0x7fb90000b870 comp rx=0 tx=0).stop 2024-05-03T20:36:22.553 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.551+0000 7fb914eef700 1 -- 172.21.15.181:0/1328495702 shutdown_connections 2024-05-03T20:36:22.553 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.551+0000 7fb914eef700 1 --2- 172.21.15.181:0/1328495702 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb8fc0b2910 0x7fb8fc0b4dd0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:22.553 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.551+0000 7fb914eef700 1 --2- 172.21.15.181:0/1328495702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9101091f0 0x7fb9101a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:22.553 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.551+0000 7fb914eef700 1 --2- 172.21.15.181:0/1328495702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb910103f90 0x7fb910199760 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:22.553 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.551+0000 7fb914eef700 1 --2- 172.21.15.181:0/1328495702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9101068c0 0x7fb910199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:22.553 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.551+0000 7fb914eef700 1 -- 172.21.15.181:0/1328495702 >> 172.21.15.181:0/1328495702 conn(0x7fb9100fd9a0 msgr2=0x7fb9101049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:22.553 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.551+0000 7fb914eef700 1 -- 172.21.15.181:0/1328495702 shutdown_connections 2024-05-03T20:36:22.553 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:22.551+0000 7fb914eef700 1 -- 172.21.15.181:0/1328495702 wait complete. 2024-05-03T20:36:23.298 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-03T20:36:23.309 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T20:36:23.551 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:36:23.673 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:23 smithi181 ceph-mon[29801]: pgmap v367: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:36:23.673 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:23 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/1328495702' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-03T20:36:23.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:23 smithi196 ceph-mon[32858]: pgmap v367: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:36:23.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:23 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/1328495702' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-03T20:36:23.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:23 smithi204 ceph-mon[33595]: pgmap v367: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:36:23.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:23 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/1328495702' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-03T20:36:25.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.296+0000 7f3acfa42700 1 -- 172.21.15.181:0/1360315810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ac81023a0 msgr2=0x7f3ac8102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:25.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.296+0000 7f3acfa42700 1 --2- 172.21.15.181:0/1360315810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ac81023a0 0x7f3ac8102820 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4009990 tx=0x7f3ac4009ca0 comp rx=0 tx=0).stop 2024-05-03T20:36:25.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.296+0000 7f3acfa42700 1 -- 172.21.15.181:0/1360315810 shutdown_connections 2024-05-03T20:36:25.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.296+0000 7f3acfa42700 1 --2- 172.21.15.181:0/1360315810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ac8102d60 0x7f3ac810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:25.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.296+0000 7f3acfa42700 1 --2- 172.21.15.181:0/1360315810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ac8108410 0x7f3ac81087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:25.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.296+0000 7f3acfa42700 1 --2- 172.21.15.181:0/1360315810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ac81023a0 0x7f3ac8102820 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:25.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.296+0000 7f3acfa42700 1 -- 172.21.15.181:0/1360315810 >> 172.21.15.181:0/1360315810 conn(0x7f3ac8079350 msgr2=0x7f3ac8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:25.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.296+0000 7f3acfa42700 1 -- 172.21.15.181:0/1360315810 shutdown_connections 2024-05-03T20:36:25.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.297+0000 7f3acfa42700 1 -- 172.21.15.181:0/1360315810 wait complete. 2024-05-03T20:36:25.298 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.298+0000 7f3acfa42700 1 Processor -- start 2024-05-03T20:36:25.298 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.298+0000 7f3acfa42700 1 -- start start 2024-05-03T20:36:25.298 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.298+0000 7f3acfa42700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ac81023a0 0x7f3ac8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:25.298 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.298+0000 7f3acfa42700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ac8102d60 0x7f3ac8198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:25.298 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.298+0000 7f3acfa42700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ac8108410 0x7f3ac81993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:25.299 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.298+0000 7f3acfa42700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac8112630 con 0x7f3ac8108410 2024-05-03T20:36:25.299 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.298+0000 7f3acfa42700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac81124b0 con 0x7f3ac8102d60 2024-05-03T20:36:25.299 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.298+0000 7f3acfa42700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac81127b0 con 0x7f3ac81023a0 2024-05-03T20:36:25.299 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.298+0000 7f3acdfdf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ac8108410 0x7f3ac81993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:25.299 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.298+0000 7f3acdfdf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ac8108410 0x7f3ac81993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34126/0 (socket says 172.21.15.181:34126) 2024-05-03T20:36:25.299 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.298+0000 7f3acdfdf700 1 -- 172.21.15.181:0/2317517998 learned_addr learned my addr 172.21.15.181:0/2317517998 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:25.299 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.298+0000 7f3accfdd700 1 --2- 172.21.15.181:0/2317517998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ac8102d60 0x7f3ac8198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:25.299 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.298+0000 7f3acd7de700 1 --2- 172.21.15.181:0/2317517998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ac81023a0 0x7f3ac8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:25.299 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.298+0000 7f3acdfdf700 1 -- 172.21.15.181:0/2317517998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ac81023a0 msgr2=0x7f3ac8198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:25.299 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.298+0000 7f3acdfdf700 1 --2- 172.21.15.181:0/2317517998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ac81023a0 0x7f3ac8198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:25.299 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.298+0000 7f3acdfdf700 1 -- 172.21.15.181:0/2317517998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ac8102d60 msgr2=0x7f3ac8198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:25.299 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.298+0000 7f3acdfdf700 1 --2- 172.21.15.181:0/2317517998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ac8102d60 0x7f3ac8198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:25.299 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.298+0000 7f3acdfdf700 1 -- 172.21.15.181:0/2317517998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac81a8190 con 0x7f3ac8108410 2024-05-03T20:36:25.299 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.299+0000 7f3acdfdf700 1 --2- 172.21.15.181:0/2317517998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ac8108410 0x7f3ac81993a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000b970 tx=0x7f3ac000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:25.299 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.299+0000 7f3abe7fc700 1 -- 172.21.15.181:0/2317517998 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac000d860 con 0x7f3ac8108410 2024-05-03T20:36:25.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.299+0000 7f3abe7fc700 1 -- 172.21.15.181:0/2317517998 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ac000e430 con 0x7f3ac8108410 2024-05-03T20:36:25.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.299+0000 7f3acfa42700 1 -- 172.21.15.181:0/2317517998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ac81a8480 con 0x7f3ac8108410 2024-05-03T20:36:25.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.299+0000 7f3abe7fc700 1 -- 172.21.15.181:0/2317517998 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0017660 con 0x7f3ac8108410 2024-05-03T20:36:25.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.299+0000 7f3acfa42700 1 -- 172.21.15.181:0/2317517998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ac81a8990 con 0x7f3ac8108410 2024-05-03T20:36:25.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.300+0000 7f3acfa42700 1 -- 172.21.15.181:0/2317517998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ac810a980 con 0x7f3ac8108410 2024-05-03T20:36:25.301 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.301+0000 7f3abe7fc700 1 -- 172.21.15.181:0/2317517998 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f3ac000da00 con 0x7f3ac8108410 2024-05-03T20:36:25.301 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.302+0000 7f3abe7fc700 1 --2- 172.21.15.181:0/2317517998 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3ab40b2910 0x7f3ab40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:25.301 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.302+0000 7f3acd7de700 1 --2- 172.21.15.181:0/2317517998 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3ab40b2910 0x7f3ab40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:25.301 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.302+0000 7f3abe7fc700 1 -- 172.21.15.181:0/2317517998 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f3ac00e1f40 con 0x7f3ac8108410 2024-05-03T20:36:25.301 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.302+0000 7f3acd7de700 1 --2- 172.21.15.181:0/2317517998 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3ab40b2910 0x7f3ab40b4dd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8009a10 tx=0x7f3ab8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:25.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.306+0000 7f3abe7fc700 1 -- 172.21.15.181:0/2317517998 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3ac0013030 con 0x7f3ac8108410 2024-05-03T20:36:25.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.566+0000 7f3acfa42700 1 -- 172.21.15.181:0/2317517998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ac806b770 con 0x7f3ac8108410 2024-05-03T20:36:25.566 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:25 smithi181 ceph-mon[29801]: pgmap v368: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:36:25.566 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:36:25.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.567+0000 7f3abe7fc700 1 -- 172.21.15.181:0/2317517998 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11954 (secure 0 0 0) 0x7f3ac00a9b00 con 0x7f3ac8108410 2024-05-03T20:36:25.567 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2414172454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2414172454},{"type":"v1","addr":"172.21.15.181:6835","nonce":2414172454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/204127189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":204127189},{"type":"v1","addr":"172.21.15.181:6837","nonce":204127189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T20:33:42.487965+0000","modified":"2024-05-03T20:34:00.359735+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24475,"mds_2":24464,"mds_3":24463,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14562":{"gid":14562,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6839/257361276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":257361276},{"type":"v1","addr":"172.21.15.181:6839","nonce":257361276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6841/2403160115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2403160115},{"type":"v1","addr":"172.21.15.181:6841","nonce":2403160115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3975688022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3975688022},{"type":"v1","addr":"172.21.15.196:6833","nonce":3975688022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2409628823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2409628823},{"type":"v1","addr":"172.21.15.204:6833","nonce":2409628823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1820206106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1820206106},{"type":"v1","addr":"172.21.15.196:6835","nonce":1820206106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6835/3870850287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3870850287},{"type":"v1","addr":"172.21.15.204:6835","nonce":3870850287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/3006728559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3006728559},{"type":"v1","addr":"172.21.15.204:6837","nonce":3006728559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6837/1529420259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1529420259},{"type":"v1","addr":"172.21.15.196:6837","nonce":1529420259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/1644828559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1644828559},{"type":"v1","addr":"172.21.15.204:6839","nonce":1644828559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.196:6839/1182826743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1182826743},{"type":"v1","addr":"172.21.15.196:6839","nonce":1182826743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24457,24464,24463,24475,24488]},"id":1}]} 2024-05-03T20:36:25.569 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.569+0000 7f3acfa42700 1 -- 172.21.15.181:0/2317517998 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3ab40b2910 msgr2=0x7f3ab40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:25.569 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.569+0000 7f3acfa42700 1 --2- 172.21.15.181:0/2317517998 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3ab40b2910 0x7f3ab40b4dd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8009a10 tx=0x7f3ab8009190 comp rx=0 tx=0).stop 2024-05-03T20:36:25.569 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.569+0000 7f3acfa42700 1 -- 172.21.15.181:0/2317517998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ac8108410 msgr2=0x7f3ac81993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:25.569 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.569+0000 7f3acfa42700 1 --2- 172.21.15.181:0/2317517998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ac8108410 0x7f3ac81993a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000b970 tx=0x7f3ac000bc80 comp rx=0 tx=0).stop 2024-05-03T20:36:25.569 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.569+0000 7f3acfa42700 1 -- 172.21.15.181:0/2317517998 shutdown_connections 2024-05-03T20:36:25.569 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.569+0000 7f3acfa42700 1 --2- 172.21.15.181:0/2317517998 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3ab40b2910 0x7f3ab40b4dd0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:25.569 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.569+0000 7f3acfa42700 1 --2- 172.21.15.181:0/2317517998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ac8108410 0x7f3ac81993a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:25.569 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.569+0000 7f3acfa42700 1 --2- 172.21.15.181:0/2317517998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ac81023a0 0x7f3ac8198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:25.569 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.569+0000 7f3acfa42700 1 --2- 172.21.15.181:0/2317517998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ac8102d60 0x7f3ac8198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:25.569 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.569+0000 7f3acfa42700 1 -- 172.21.15.181:0/2317517998 >> 172.21.15.181:0/2317517998 conn(0x7f3ac8079350 msgr2=0x7f3ac8101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:25.570 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.570+0000 7f3acfa42700 1 -- 172.21.15.181:0/2317517998 shutdown_connections 2024-05-03T20:36:25.570 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:25.570+0000 7f3acfa42700 1 -- 172.21.15.181:0/2317517998 wait complete. 2024-05-03T20:36:25.570 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 27 2024-05-03T20:36:25.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:25 smithi196 ceph-mon[32858]: pgmap v368: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:36:25.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:25 smithi204 ceph-mon[33595]: pgmap v368: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:36:26.295 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T20:36:26.536 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:36:26.658 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:26 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/2317517998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:36:26.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:26 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/2317517998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:36:26.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:26 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/2317517998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:36:27.700 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:27 smithi181 ceph-mon[29801]: pgmap v369: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:36:27.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:27 smithi196 ceph-mon[32858]: pgmap v369: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:36:27.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:27 smithi204 ceph-mon[33595]: pgmap v369: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:36:28.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.266+0000 7fe4e6943700 1 -- 172.21.15.181:0/2963515239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4e01023a0 msgr2=0x7fe4e0102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:28.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.266+0000 7fe4e6943700 1 --2- 172.21.15.181:0/2963515239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4e01023a0 0x7fe4e0102820 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009990 tx=0x7fe4d0009ca0 comp rx=0 tx=0).stop 2024-05-03T20:36:28.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.266+0000 7fe4e6943700 1 -- 172.21.15.181:0/2963515239 shutdown_connections 2024-05-03T20:36:28.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.266+0000 7fe4e6943700 1 --2- 172.21.15.181:0/2963515239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4e0102d60 0x7fe4e010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:28.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.266+0000 7fe4e6943700 1 --2- 172.21.15.181:0/2963515239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4e0108410 0x7fe4e01087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:28.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.266+0000 7fe4e6943700 1 --2- 172.21.15.181:0/2963515239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4e01023a0 0x7fe4e0102820 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:28.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.266+0000 7fe4e6943700 1 -- 172.21.15.181:0/2963515239 >> 172.21.15.181:0/2963515239 conn(0x7fe4e0079350 msgr2=0x7fe4e0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:28.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.266+0000 7fe4e6943700 1 -- 172.21.15.181:0/2963515239 shutdown_connections 2024-05-03T20:36:28.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.266+0000 7fe4e6943700 1 -- 172.21.15.181:0/2963515239 wait complete. 2024-05-03T20:36:28.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.266+0000 7fe4e6943700 1 Processor -- start 2024-05-03T20:36:28.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.267+0000 7fe4e6943700 1 -- start start 2024-05-03T20:36:28.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.267+0000 7fe4e6943700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4e01023a0 0x7fe4e0104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:28.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.267+0000 7fe4e6943700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4e0102d60 0x7fe4e01045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:28.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.267+0000 7fe4e6943700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4e0108410 0x7fe4e0104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:28.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.267+0000 7fe4e6943700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e0112630 con 0x7fe4e0108410 2024-05-03T20:36:28.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.267+0000 7fe4e6943700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e01124b0 con 0x7fe4e01023a0 2024-05-03T20:36:28.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.267+0000 7fe4e6943700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e01127b0 con 0x7fe4e0102d60 2024-05-03T20:36:28.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.267+0000 7fe4e4ee0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4e0108410 0x7fe4e0104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:28.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.267+0000 7fe4e4ee0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4e0108410 0x7fe4e0104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45178/0 (socket says 172.21.15.181:45178) 2024-05-03T20:36:28.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.267+0000 7fe4e4ee0700 1 -- 172.21.15.181:0/1562659294 learned_addr learned my addr 172.21.15.181:0/1562659294 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:28.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.267+0000 7fe4dffff700 1 --2- 172.21.15.181:0/1562659294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4e01023a0 0x7fe4e0104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:28.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.267+0000 7fe4df7fe700 1 --2- 172.21.15.181:0/1562659294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4e0102d60 0x7fe4e01045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:28.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.267+0000 7fe4e4ee0700 1 -- 172.21.15.181:0/1562659294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4e0102d60 msgr2=0x7fe4e01045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:28.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.267+0000 7fe4e4ee0700 1 --2- 172.21.15.181:0/1562659294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4e0102d60 0x7fe4e01045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:28.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.267+0000 7fe4e4ee0700 1 -- 172.21.15.181:0/1562659294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4e01023a0 msgr2=0x7fe4e0104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:28.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.267+0000 7fe4e4ee0700 1 --2- 172.21.15.181:0/1562659294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4e01023a0 0x7fe4e0104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:28.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.267+0000 7fe4e4ee0700 1 -- 172.21.15.181:0/1562659294 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e01a8220 con 0x7fe4e0108410 2024-05-03T20:36:28.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.268+0000 7fe4e4ee0700 1 --2- 172.21.15.181:0/1562659294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4e0108410 0x7fe4e0104b00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800b970 tx=0x7fe4d800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:28.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.268+0000 7fe4dd7fa700 1 -- 172.21.15.181:0/1562659294 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d800d860 con 0x7fe4e0108410 2024-05-03T20:36:28.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.268+0000 7fe4dd7fa700 1 -- 172.21.15.181:0/1562659294 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d800e430 con 0x7fe4e0108410 2024-05-03T20:36:28.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.268+0000 7fe4e6943700 1 -- 172.21.15.181:0/1562659294 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e01a8510 con 0x7fe4e0108410 2024-05-03T20:36:28.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.268+0000 7fe4dd7fa700 1 -- 172.21.15.181:0/1562659294 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8017660 con 0x7fe4e0108410 2024-05-03T20:36:28.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.268+0000 7fe4e6943700 1 -- 172.21.15.181:0/1562659294 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e01a8a20 con 0x7fe4e0108410 2024-05-03T20:36:28.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.269+0000 7fe4e6943700 1 -- 172.21.15.181:0/1562659294 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4e0103ac0 con 0x7fe4e0108410 2024-05-03T20:36:28.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.270+0000 7fe4dd7fa700 1 -- 172.21.15.181:0/1562659294 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fe4d800da00 con 0x7fe4e0108410 2024-05-03T20:36:28.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.271+0000 7fe4dd7fa700 1 --2- 172.21.15.181:0/1562659294 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fe4c80b2910 0x7fe4c80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:28.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.271+0000 7fe4dffff700 1 --2- 172.21.15.181:0/1562659294 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fe4c80b2910 0x7fe4c80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:28.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.271+0000 7fe4dd7fa700 1 -- 172.21.15.181:0/1562659294 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7fe4d80e23e0 con 0x7fe4e0108410 2024-05-03T20:36:28.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.272+0000 7fe4dffff700 1 --2- 172.21.15.181:0/1562659294 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fe4c80b2910 0x7fe4c80b4dd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009a10 tx=0x7fe4d4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:28.274 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.275+0000 7fe4dd7fa700 1 -- 172.21.15.181:0/1562659294 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe4d8013030 con 0x7fe4e0108410 2024-05-03T20:36:28.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.538+0000 7fe4e6943700 1 -- 172.21.15.181:0/1562659294 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4e006a190 con 0x7fe4e0108410 2024-05-03T20:36:28.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.539+0000 7fe4dd7fa700 1 -- 172.21.15.181:0/1562659294 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11954 (secure 0 0 0) 0x7fe4e006a190 con 0x7fe4e0108410 2024-05-03T20:36:28.538 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:36:28.539 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2414172454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2414172454},{"type":"v1","addr":"172.21.15.181:6835","nonce":2414172454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/204127189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":204127189},{"type":"v1","addr":"172.21.15.181:6837","nonce":204127189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T20:33:42.487965+0000","modified":"2024-05-03T20:34:00.359735+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24475,"mds_2":24464,"mds_3":24463,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14562":{"gid":14562,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6839/257361276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":257361276},{"type":"v1","addr":"172.21.15.181:6839","nonce":257361276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6841/2403160115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2403160115},{"type":"v1","addr":"172.21.15.181:6841","nonce":2403160115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3975688022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3975688022},{"type":"v1","addr":"172.21.15.196:6833","nonce":3975688022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2409628823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2409628823},{"type":"v1","addr":"172.21.15.204:6833","nonce":2409628823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1820206106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1820206106},{"type":"v1","addr":"172.21.15.196:6835","nonce":1820206106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6835/3870850287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3870850287},{"type":"v1","addr":"172.21.15.204:6835","nonce":3870850287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/3006728559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3006728559},{"type":"v1","addr":"172.21.15.204:6837","nonce":3006728559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6837/1529420259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1529420259},{"type":"v1","addr":"172.21.15.196:6837","nonce":1529420259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/1644828559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1644828559},{"type":"v1","addr":"172.21.15.204:6839","nonce":1644828559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.196:6839/1182826743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1182826743},{"type":"v1","addr":"172.21.15.196:6839","nonce":1182826743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24457,24464,24463,24475,24488]},"id":1}]} 2024-05-03T20:36:28.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.541+0000 7fe4e6943700 1 -- 172.21.15.181:0/1562659294 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fe4c80b2910 msgr2=0x7fe4c80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:28.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.541+0000 7fe4e6943700 1 --2- 172.21.15.181:0/1562659294 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fe4c80b2910 0x7fe4c80b4dd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009a10 tx=0x7fe4d4009190 comp rx=0 tx=0).stop 2024-05-03T20:36:28.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.541+0000 7fe4e6943700 1 -- 172.21.15.181:0/1562659294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4e0108410 msgr2=0x7fe4e0104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:28.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.541+0000 7fe4e6943700 1 --2- 172.21.15.181:0/1562659294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4e0108410 0x7fe4e0104b00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800b970 tx=0x7fe4d800bc80 comp rx=0 tx=0).stop 2024-05-03T20:36:28.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.541+0000 7fe4e6943700 1 -- 172.21.15.181:0/1562659294 shutdown_connections 2024-05-03T20:36:28.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.541+0000 7fe4e6943700 1 --2- 172.21.15.181:0/1562659294 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fe4c80b2910 0x7fe4c80b4dd0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:28.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.541+0000 7fe4e6943700 1 --2- 172.21.15.181:0/1562659294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4e0108410 0x7fe4e0104b00 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:28.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.541+0000 7fe4e6943700 1 --2- 172.21.15.181:0/1562659294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4e01023a0 0x7fe4e0104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:28.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.541+0000 7fe4e6943700 1 --2- 172.21.15.181:0/1562659294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4e0102d60 0x7fe4e01045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:28.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.541+0000 7fe4e6943700 1 -- 172.21.15.181:0/1562659294 >> 172.21.15.181:0/1562659294 conn(0x7fe4e0079350 msgr2=0x7fe4e01017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:28.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.541+0000 7fe4e6943700 1 -- 172.21.15.181:0/1562659294 shutdown_connections 2024-05-03T20:36:28.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:28.541+0000 7fe4e6943700 1 -- 172.21.15.181:0/1562659294 wait complete. 2024-05-03T20:36:28.542 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 27 2024-05-03T20:36:29.151 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T20:36:29.402 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:36:29.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:29 smithi181 ceph-mon[29801]: pgmap v370: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:36:29.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:29 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/1562659294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:36:29.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:29 smithi196 ceph-mon[32858]: pgmap v370: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:36:29.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:29 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/1562659294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:36:29.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:29 smithi204 ceph-mon[33595]: pgmap v370: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:36:29.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:29 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/1562659294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:36:31.039 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.037+0000 7fcf1a891700 1 -- 172.21.15.181:0/802949981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf14103750 msgr2=0x7fcf14103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:31.039 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.037+0000 7fcf1a891700 1 --2- 172.21.15.181:0/802949981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf14103750 0x7fcf14103b30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcf040099c0 tx=0x7fcf04009cd0 comp rx=0 tx=0).stop 2024-05-03T20:36:31.040 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.037+0000 7fcf1a891700 1 -- 172.21.15.181:0/802949981 shutdown_connections 2024-05-03T20:36:31.040 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.037+0000 7fcf1a891700 1 --2- 172.21.15.181:0/802949981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf14108850 0x7fcf141103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:31.040 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.037+0000 7fcf1a891700 1 --2- 172.21.15.181:0/802949981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf14103750 0x7fcf14103b30 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:31.040 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.037+0000 7fcf1a891700 1 --2- 172.21.15.181:0/802949981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf14104100 0x7fcf14108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:31.040 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.037+0000 7fcf1a891700 1 -- 172.21.15.181:0/802949981 >> 172.21.15.181:0/802949981 conn(0x7fcf1407a600 msgr2=0x7fcf1407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:31.040 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.037+0000 7fcf1a891700 1 -- 172.21.15.181:0/802949981 shutdown_connections 2024-05-03T20:36:31.040 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.037+0000 7fcf1a891700 1 -- 172.21.15.181:0/802949981 wait complete. 2024-05-03T20:36:31.040 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.038+0000 7fcf1a891700 1 Processor -- start 2024-05-03T20:36:31.040 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.038+0000 7fcf1a891700 1 -- start start 2024-05-03T20:36:31.040 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.038+0000 7fcf1a891700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf14103750 0x7fcf1419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:31.040 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.038+0000 7fcf1a891700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf14104100 0x7fcf1419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:31.040 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.038+0000 7fcf1a891700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf14108850 0x7fcf14199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:31.040 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.038+0000 7fcf1a891700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141136a0 con 0x7fcf14103750 2024-05-03T20:36:31.040 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.038+0000 7fcf1a891700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf14113520 con 0x7fcf14104100 2024-05-03T20:36:31.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.038+0000 7fcf1a891700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcf14113820 con 0x7fcf14108850 2024-05-03T20:36:31.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.038+0000 7fcf13fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf14103750 0x7fcf1419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:31.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.038+0000 7fcf13fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf14103750 0x7fcf1419f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45190/0 (socket says 172.21.15.181:45190) 2024-05-03T20:36:31.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.038+0000 7fcf13fff700 1 -- 172.21.15.181:0/3267452848 learned_addr learned my addr 172.21.15.181:0/3267452848 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:31.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.038+0000 7fcf137fe700 1 --2- 172.21.15.181:0/3267452848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf14104100 0x7fcf1419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:31.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.038+0000 7fcf18e2e700 1 --2- 172.21.15.181:0/3267452848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf14108850 0x7fcf14199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:31.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.039+0000 7fcf13fff700 1 -- 172.21.15.181:0/3267452848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf14108850 msgr2=0x7fcf14199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:31.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.039+0000 7fcf13fff700 1 --2- 172.21.15.181:0/3267452848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf14108850 0x7fcf14199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:31.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.039+0000 7fcf13fff700 1 -- 172.21.15.181:0/3267452848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf14104100 msgr2=0x7fcf1419fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:31.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.039+0000 7fcf13fff700 1 --2- 172.21.15.181:0/3267452848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf14104100 0x7fcf1419fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:31.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.039+0000 7fcf13fff700 1 -- 172.21.15.181:0/3267452848 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1419a080 con 0x7fcf14103750 2024-05-03T20:36:31.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.039+0000 7fcf13fff700 1 --2- 172.21.15.181:0/3267452848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf14103750 0x7fcf1419f710 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400b580 tx=0x7fcf040047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:31.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.039+0000 7fcf117fa700 1 -- 172.21.15.181:0/3267452848 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf04027070 con 0x7fcf14103750 2024-05-03T20:36:31.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.039+0000 7fcf117fa700 1 -- 172.21.15.181:0/3267452848 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf04004020 con 0x7fcf14103750 2024-05-03T20:36:31.042 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.039+0000 7fcf1a891700 1 -- 172.21.15.181:0/3267452848 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf1419a310 con 0x7fcf14103750 2024-05-03T20:36:31.042 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.039+0000 7fcf117fa700 1 -- 172.21.15.181:0/3267452848 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf04022e10 con 0x7fcf14103750 2024-05-03T20:36:31.042 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.039+0000 7fcf1a891700 1 -- 172.21.15.181:0/3267452848 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf141a65e0 con 0x7fcf14103750 2024-05-03T20:36:31.042 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.040+0000 7fcf1a891700 1 -- 172.21.15.181:0/3267452848 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf14104be0 con 0x7fcf14103750 2024-05-03T20:36:31.043 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.041+0000 7fcf117fa700 1 -- 172.21.15.181:0/3267452848 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fcf0402a070 con 0x7fcf14103750 2024-05-03T20:36:31.043 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.042+0000 7fcf117fa700 1 --2- 172.21.15.181:0/3267452848 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fcefc0b2910 0x7fcefc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:31.043 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.042+0000 7fcf137fe700 1 --2- 172.21.15.181:0/3267452848 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fcefc0b2910 0x7fcefc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:31.043 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.042+0000 7fcf117fa700 1 -- 172.21.15.181:0/3267452848 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7fcf040e3120 con 0x7fcf14103750 2024-05-03T20:36:31.043 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.042+0000 7fcf137fe700 1 --2- 172.21.15.181:0/3267452848 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fcefc0b2910 0x7fcefc0b4dd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcf14103030 tx=0x7fcf0800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:31.046 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.046+0000 7fcf117fa700 1 -- 172.21.15.181:0/3267452848 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fcf04013070 con 0x7fcf14103750 2024-05-03T20:36:31.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.256+0000 7fcf1a891700 1 -- 172.21.15.181:0/3267452848 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf1419ae00 con 0x7fcf14103750 2024-05-03T20:36:31.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.257+0000 7fcf117fa700 1 -- 172.21.15.181:0/3267452848 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22710 (secure 0 0 0) 0x7fcf040aad70 con 0x7fcf14103750 2024-05-03T20:36:31.257 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:36:31.258 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":83,"fsid":"1cd058ee-098b-11ef-bc97-c7b262605968","created":"2024-05-03T20:25:39.427815+0000","modified":"2024-05-03T20:34:35.581238+0000","last_up_change":"2024-05-03T20:32:38.854730+0000","last_in_change":"2024-05-03T20:32:18.787388+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T20:29:01.855187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T20:33:35.737273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T20:33:39.337556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5f594d97-10a4-4f77-aebc-c00dbe92a4af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6803","nonce":1683973278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6805","nonce":1683973278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6809","nonce":1683973278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6807","nonce":1683973278}]},"public_addr":"172.21.15.181:6803/1683973278","cluster_addr":"172.21.15.181:6805/1683973278","heartbeat_back_addr":"172.21.15.181:6809/1683973278","heartbeat_front_addr":"172.21.15.181:6807/1683973278","state":["exists","up"]},{"osd":1,"uuid":"9726d641-8930-4b45-9192-6ee2b6bfcdcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6801","nonce":2136873375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6803","nonce":2136873375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6807","nonce":2136873375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6805","nonce":2136873375}]},"public_addr":"172.21.15.196:6801/2136873375","cluster_addr":"172.21.15.196:6803/2136873375","heartbeat_back_addr":"172.21.15.196:6807/2136873375","heartbeat_front_addr":"172.21.15.196:6805/2136873375","state":["exists","up"]},{"osd":2,"uuid":"3d5573cc-37c1-4c3b-a61e-2363ef3ff964","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6801","nonce":3291673137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6803","nonce":3291673137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6807","nonce":3291673137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6805","nonce":3291673137}]},"public_addr":"172.21.15.204:6801/3291673137","cluster_addr":"172.21.15.204:6803/3291673137","heartbeat_back_addr":"172.21.15.204:6807/3291673137","heartbeat_front_addr":"172.21.15.204:6805/3291673137","state":["exists","up"]},{"osd":3,"uuid":"b9b205e3-a335-4771-9a66-55cfe6a8c31a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6811","nonce":29064971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6813","nonce":29064971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6817","nonce":29064971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6815","nonce":29064971}]},"public_addr":"172.21.15.181:6811/29064971","cluster_addr":"172.21.15.181:6813/29064971","heartbeat_back_addr":"172.21.15.181:6817/29064971","heartbeat_front_addr":"172.21.15.181:6815/29064971","state":["exists","up"]},{"osd":4,"uuid":"550f01ef-1947-4f9f-b759-34e3ea831839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6809","nonce":1145408995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6811","nonce":1145408995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6815","nonce":1145408995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6813","nonce":1145408995}]},"public_addr":"172.21.15.196:6809/1145408995","cluster_addr":"172.21.15.196:6811/1145408995","heartbeat_back_addr":"172.21.15.196:6815/1145408995","heartbeat_front_addr":"172.21.15.196:6813/1145408995","state":["exists","up"]},{"osd":5,"uuid":"6b74cb3a-f39a-4131-8793-1d5286856aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6809","nonce":855680559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6811","nonce":855680559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6815","nonce":855680559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6813","nonce":855680559}]},"public_addr":"172.21.15.204:6809/855680559","cluster_addr":"172.21.15.204:6811/855680559","heartbeat_back_addr":"172.21.15.204:6815/855680559","heartbeat_front_addr":"172.21.15.204:6813/855680559","state":["exists","up"]},{"osd":6,"uuid":"dbe19bdb-08eb-40aa-842b-0710963d4497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6819","nonce":1561969338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6821","nonce":1561969338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6825","nonce":1561969338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6823","nonce":1561969338}]},"public_addr":"172.21.15.181:6819/1561969338","cluster_addr":"172.21.15.181:6821/1561969338","heartbeat_back_addr":"172.21.15.181:6825/1561969338","heartbeat_front_addr":"172.21.15.181:6823/1561969338","state":["exists","up"]},{"osd":7,"uuid":"016c145c-ad67-42ac-9775-ca2840790abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6817","nonce":824890163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6819","nonce":824890163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6823","nonce":824890163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6821","nonce":824890163}]},"public_addr":"172.21.15.196:6817/824890163","cluster_addr":"172.21.15.196:6819/824890163","heartbeat_back_addr":"172.21.15.196:6823/824890163","heartbeat_front_addr":"172.21.15.196:6821/824890163","state":["exists","up"]},{"osd":8,"uuid":"10235638-c1f1-41d3-8e24-f8b2344bfeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6817","nonce":3566909552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6819","nonce":3566909552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6823","nonce":3566909552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6821","nonce":3566909552}]},"public_addr":"172.21.15.204:6817/3566909552","cluster_addr":"172.21.15.204:6819/3566909552","heartbeat_back_addr":"172.21.15.204:6823/3566909552","heartbeat_front_addr":"172.21.15.204:6821/3566909552","state":["exists","up"]},{"osd":9,"uuid":"96ce5aa6-e6a9-4d83-8562-589ac94c8cd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6827","nonce":241693472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6829","nonce":241693472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6833","nonce":241693472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6831","nonce":241693472}]},"public_addr":"172.21.15.181:6827/241693472","cluster_addr":"172.21.15.181:6829/241693472","heartbeat_back_addr":"172.21.15.181:6833/241693472","heartbeat_front_addr":"172.21.15.181:6831/241693472","state":["exists","up"]},{"osd":10,"uuid":"d6c9ffab-731c-46d5-88cd-5f21ba5acfa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6825","nonce":3980645995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6827","nonce":3980645995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6831","nonce":3980645995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6829","nonce":3980645995}]},"public_addr":"172.21.15.196:6825/3980645995","cluster_addr":"172.21.15.196:6827/3980645995","heartbeat_back_addr":"172.21.15.196:6831/3980645995","heartbeat_front_addr":"172.21.15.196:6829/3980645995","state":["exists","up"]},{"osd":11,"uuid":"3f58a2d6-bbbb-4e4d-bf83-2c886dd9484b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6825","nonce":488710574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6827","nonce":488710574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6831","nonce":488710574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6829","nonce":488710574}]},"public_addr":"172.21.15.204:6825/488710574","cluster_addr":"172.21.15.204:6827/488710574","heartbeat_back_addr":"172.21.15.204:6831/488710574","heartbeat_front_addr":"172.21.15.204:6829/488710574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:28:09.700091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:28:33.838568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:28:57.265536+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:29:21.720066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:29:45.480447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:30:09.558384+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:30:34.253331+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:30:57.986111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:31:22.470746+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:31:47.548762+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:32:11.285657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:32:36.888004+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":6},{"from":7,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6800/2389231557":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/3398276284":"2024-05-04T20:26:06.561713+0000","172.21.15.181:6801/2389231557":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/2327733609":"2024-05-04T20:26:06.561713+0000","172.21.15.181:0/628540759":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/3387163329":"2024-05-04T20:26:06.561713+0000","172.21.15.181:6800/1802984663":"2024-05-04T20:26:06.561713+0000","172.21.15.181:0/67924109":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/3693389657":"2024-05-04T20:26:33.766772+0000","172.21.15.181:6801/1802984663":"2024-05-04T20:26:06.561713+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T20:36:31.259 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.260+0000 7fcf1a891700 1 -- 172.21.15.181:0/3267452848 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fcefc0b2910 msgr2=0x7fcefc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:31.259 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.260+0000 7fcf1a891700 1 --2- 172.21.15.181:0/3267452848 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fcefc0b2910 0x7fcefc0b4dd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcf14103030 tx=0x7fcf0800a380 comp rx=0 tx=0).stop 2024-05-03T20:36:31.259 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.260+0000 7fcf1a891700 1 -- 172.21.15.181:0/3267452848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf14103750 msgr2=0x7fcf1419f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:31.259 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.260+0000 7fcf1a891700 1 --2- 172.21.15.181:0/3267452848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf14103750 0x7fcf1419f710 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400b580 tx=0x7fcf040047b0 comp rx=0 tx=0).stop 2024-05-03T20:36:31.260 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.260+0000 7fcf1a891700 1 -- 172.21.15.181:0/3267452848 shutdown_connections 2024-05-03T20:36:31.260 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.260+0000 7fcf1a891700 1 --2- 172.21.15.181:0/3267452848 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fcefc0b2910 0x7fcefc0b4dd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:31.260 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.260+0000 7fcf1a891700 1 --2- 172.21.15.181:0/3267452848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf14108850 0x7fcf14199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:31.260 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.260+0000 7fcf1a891700 1 --2- 172.21.15.181:0/3267452848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf14103750 0x7fcf1419f710 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:31.260 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.260+0000 7fcf1a891700 1 --2- 172.21.15.181:0/3267452848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf14104100 0x7fcf1419fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:31.260 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.260+0000 7fcf1a891700 1 -- 172.21.15.181:0/3267452848 >> 172.21.15.181:0/3267452848 conn(0x7fcf1407a600 msgr2=0x7fcf141016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:31.260 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.260+0000 7fcf1a891700 1 -- 172.21.15.181:0/3267452848 shutdown_connections 2024-05-03T20:36:31.260 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:31.260+0000 7fcf1a891700 1 -- 172.21.15.181:0/3267452848 wait complete. 2024-05-03T20:36:31.622 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:31 smithi181 ceph-mon[29801]: pgmap v371: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:36:31.622 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:31 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3267452848' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T20:36:31.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:31 smithi196 ceph-mon[32858]: pgmap v371: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:36:31.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:31 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3267452848' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T20:36:31.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:31 smithi204 ceph-mon[33595]: pgmap v371: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:36:31.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:31 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3267452848' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T20:36:31.928 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-03T20:33:42.487965+0000', 'modified': '2024-05-03T20:34:00.359735+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24488, 'mds_1': 24475, 'mds_2': 24464, 'mds_3': 24463, 'mds_4': 24457}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14562': {'gid': 14562, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6839/257361276', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 257361276}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 257361276}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14574': {'gid': 14574, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6841/2403160115', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6840', 'nonce': 2403160115}, {'type': 'v1', 'addr': '172.21.15.181:6841', 'nonce': 2403160115}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24410': {'gid': 24410, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6833/3975688022', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 3975688022}, {'type': 'v1', 'addr': '172.21.15.196:6833', 'nonce': 3975688022}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6833/2409628823', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6832', 'nonce': 2409628823}, {'type': 'v1', 'addr': '172.21.15.204:6833', 'nonce': 2409628823}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24437': {'gid': 24437, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6835/1820206106', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 1820206106}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 1820206106}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.204:6835/3870850287', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 3870850287}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 3870850287}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.204:6837/3006728559', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 3006728559}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 3006728559}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24464': {'gid': 24464, 'name': 'h', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.196:6837/1529420259', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6836', 'nonce': 1529420259}, {'type': 'v1', 'addr': '172.21.15.196:6837', 'nonce': 1529420259}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.204:6839/1644828559', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 1644828559}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 1644828559}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.196:6839/1182826743', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6838', 'nonce': 1182826743}, {'type': 'v1', 'addr': '172.21.15.196:6839', 'nonce': 1182826743}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24488, 'qdb_cluster': [24457, 24464, 24463, 24475, 24488]} 2024-05-03T20:36:31.928 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-03T20:36:31.928 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph tell mds.f status 2024-05-03T20:36:32.170 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:36:33.698 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:33 smithi181 ceph-mon[29801]: pgmap v372: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:36:33.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:33 smithi196 ceph-mon[32858]: pgmap v372: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:36:33.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:33 smithi204 ceph-mon[33595]: pgmap v372: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:36:33.938 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.939+0000 7fed40bbb700 1 -- 172.21.15.181:0/2007192415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed3c108400 msgr2=0x7fed3c1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:33.938 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.939+0000 7fed40bbb700 1 --2- 172.21.15.181:0/2007192415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed3c108400 0x7fed3c1087e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fed2c009990 tx=0x7fed2c009ca0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.938 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.939+0000 7fed40bbb700 1 -- 172.21.15.181:0/2007192415 shutdown_connections 2024-05-03T20:36:33.938 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.939+0000 7fed40bbb700 1 --2- 172.21.15.181:0/2007192415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed3c102d50 0x7fed3c10f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.939 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.939+0000 7fed40bbb700 1 --2- 172.21.15.181:0/2007192415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed3c108400 0x7fed3c1087e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.939 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.939+0000 7fed40bbb700 1 --2- 172.21.15.181:0/2007192415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed3c102390 0x7fed3c102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.939 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.939+0000 7fed40bbb700 1 -- 172.21.15.181:0/2007192415 >> 172.21.15.181:0/2007192415 conn(0x7fed3c079310 msgr2=0x7fed3c079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:33.939 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.939+0000 7fed40bbb700 1 -- 172.21.15.181:0/2007192415 shutdown_connections 2024-05-03T20:36:33.939 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.939+0000 7fed40bbb700 1 -- 172.21.15.181:0/2007192415 wait complete. 2024-05-03T20:36:33.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.939+0000 7fed40bbb700 1 Processor -- start 2024-05-03T20:36:33.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.940+0000 7fed40bbb700 1 -- start start 2024-05-03T20:36:33.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.940+0000 7fed40bbb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed3c102390 0x7fed3c104070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:33.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.940+0000 7fed40bbb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed3c102d50 0x7fed3c1045b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:33.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.940+0000 7fed40bbb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed3c108400 0x7fed3c104af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:33.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.940+0000 7fed40bbb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c112620 con 0x7fed3c108400 2024-05-03T20:36:33.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.940+0000 7fed40bbb700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1124a0 con 0x7fed3c102390 2024-05-03T20:36:33.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.940+0000 7fed40bbb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1127a0 con 0x7fed3c102d50 2024-05-03T20:36:33.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.940+0000 7fed3ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed3c108400 0x7fed3c104af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:33.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.940+0000 7fed3ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed3c108400 0x7fed3c104af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45210/0 (socket says 172.21.15.181:45210) 2024-05-03T20:36:33.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.940+0000 7fed3ad9d700 1 -- 172.21.15.181:0/2002080116 learned_addr learned my addr 172.21.15.181:0/2002080116 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:33.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.940+0000 7fed3a59c700 1 --2- 172.21.15.181:0/2002080116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed3c102390 0x7fed3c104070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:33.942 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.940+0000 7fed31d9b700 1 --2- 172.21.15.181:0/2002080116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed3c102d50 0x7fed3c1045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:33.942 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.940+0000 7fed3ad9d700 1 -- 172.21.15.181:0/2002080116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed3c102d50 msgr2=0x7fed3c1045b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:33.942 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.940+0000 7fed3ad9d700 1 --2- 172.21.15.181:0/2002080116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed3c102d50 0x7fed3c1045b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.942 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.940+0000 7fed3ad9d700 1 -- 172.21.15.181:0/2002080116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed3c102390 msgr2=0x7fed3c104070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:33.942 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.940+0000 7fed3ad9d700 1 --2- 172.21.15.181:0/2002080116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed3c102390 0x7fed3c104070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.942 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.940+0000 7fed3ad9d700 1 -- 172.21.15.181:0/2002080116 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed3c1a8240 con 0x7fed3c108400 2024-05-03T20:36:33.942 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.940+0000 7fed31d9b700 1 --2- 172.21.15.181:0/2002080116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed3c102d50 0x7fed3c1045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:36:33.942 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.940+0000 7fed3ad9d700 1 --2- 172.21.15.181:0/2002080116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed3c108400 0x7fed3c104af0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fed3400b970 tx=0x7fed3400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:33.943 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.941+0000 7fed33fff700 1 -- 172.21.15.181:0/2002080116 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34019070 con 0x7fed3c108400 2024-05-03T20:36:33.943 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.941+0000 7fed40bbb700 1 -- 172.21.15.181:0/2002080116 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed3c1a8530 con 0x7fed3c108400 2024-05-03T20:36:33.943 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.941+0000 7fed33fff700 1 -- 172.21.15.181:0/2002080116 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed3400cde0 con 0x7fed3c108400 2024-05-03T20:36:33.943 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.941+0000 7fed33fff700 1 -- 172.21.15.181:0/2002080116 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34007370 con 0x7fed3c108400 2024-05-03T20:36:33.943 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.941+0000 7fed40bbb700 1 -- 172.21.15.181:0/2002080116 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed3c1a8a70 con 0x7fed3c108400 2024-05-03T20:36:33.946 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.943+0000 7fed33fff700 1 -- 172.21.15.181:0/2002080116 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fed3401c070 con 0x7fed3c108400 2024-05-03T20:36:33.946 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.943+0000 7fed33fff700 1 --2- 172.21.15.181:0/2002080116 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fed280b2910 0x7fed280b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:33.946 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.944+0000 7fed3a59c700 1 --2- 172.21.15.181:0/2002080116 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fed280b2910 0x7fed280b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:33.946 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.944+0000 7fed33fff700 1 -- 172.21.15.181:0/2002080116 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7fed340e17c0 con 0x7fed3c108400 2024-05-03T20:36:33.946 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.944+0000 7fed3a59c700 1 --2- 172.21.15.181:0/2002080116 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fed280b2910 0x7fed280b4dd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00b5c0 tx=0x7fed2c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:33.946 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.944+0000 7fed4b44eb80 1 Processor -- start 2024-05-03T20:36:33.946 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.944+0000 7fed4b44eb80 1 -- start start 2024-05-03T20:36:33.946 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.945+0000 7fed4b44eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d02ae740 0x55b5d02aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:33.946 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.945+0000 7fed4b44eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5d02af060 0x55b5d02a9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:33.946 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.945+0000 7fed4b44eb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02aa480 0x55b5d02aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:33.946 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.945+0000 7fed4b44eb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b5d01fc8f0 con 0x55b5d02aa480 2024-05-03T20:36:33.946 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.945+0000 7fed4b44eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b5d0161d50 con 0x55b5d02ae740 2024-05-03T20:36:33.946 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.945+0000 7fed4b44eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b5d01f88b0 con 0x55b5d02af060 2024-05-03T20:36:33.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.945+0000 7fed3ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02aa480 0x55b5d02aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:33.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.945+0000 7fed3ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02aa480 0x55b5d02aa8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45214/0 (socket says 172.21.15.181:45214) 2024-05-03T20:36:33.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.945+0000 7fed3ad9d700 1 -- 172.21.15.181:0/919187379 learned_addr learned my addr 172.21.15.181:0/919187379 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:33.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.945+0000 7fed3a59c700 1 --2- 172.21.15.181:0/919187379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d02ae740 0x55b5d02aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:33.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.945+0000 7fed31d9b700 1 --2- 172.21.15.181:0/919187379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5d02af060 0x55b5d02a9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:33.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.945+0000 7fed3ad9d700 1 -- 172.21.15.181:0/919187379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5d02af060 msgr2=0x55b5d02a9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:33.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.945+0000 7fed3ad9d700 1 --2- 172.21.15.181:0/919187379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5d02af060 0x55b5d02a9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.945+0000 7fed3ad9d700 1 -- 172.21.15.181:0/919187379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d02ae740 msgr2=0x55b5d02aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:33.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.945+0000 7fed3ad9d700 1 --2- 172.21.15.181:0/919187379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d02ae740 0x55b5d02aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.945+0000 7fed3ad9d700 1 -- 172.21.15.181:0/919187379 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5d0149880 con 0x55b5d02aa480 2024-05-03T20:36:33.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.945+0000 7fed3ad9d700 1 --2- 172.21.15.181:0/919187379 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02aa480 0x55b5d02aa8c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fed340a8a90 tx=0x7fed340a8610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f678469acc3393 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:33.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.945+0000 7fed30d99700 1 -- 172.21.15.181:0/919187379 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed340ad140 con 0x55b5d02aa480 2024-05-03T20:36:33.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.945+0000 7fed30d99700 1 -- 172.21.15.181:0/919187379 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed340b1bf0 con 0x55b5d02aa480 2024-05-03T20:36:33.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.945+0000 7fed30d99700 1 -- 172.21.15.181:0/919187379 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed340b19c0 con 0x55b5d02aa480 2024-05-03T20:36:33.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.946+0000 7fed4b44eb80 1 -- 172.21.15.181:0/919187379 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02aa480 msgr2=0x55b5d02aa8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:33.948 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.946+0000 7fed4b44eb80 1 --2- 172.21.15.181:0/919187379 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02aa480 0x55b5d02aa8c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fed340a8a90 tx=0x7fed340a8610 comp rx=0 tx=0).stop 2024-05-03T20:36:33.948 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.946+0000 7fed4b44eb80 1 -- 172.21.15.181:0/919187379 shutdown_connections 2024-05-03T20:36:33.948 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.946+0000 7fed4b44eb80 1 --2- 172.21.15.181:0/919187379 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02aa480 0x55b5d02aa8c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.948 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.946+0000 7fed4b44eb80 1 --2- 172.21.15.181:0/919187379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d02ae740 0x55b5d02aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.948 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.946+0000 7fed4b44eb80 1 --2- 172.21.15.181:0/919187379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5d02af060 0x55b5d02a9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.948 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.946+0000 7fed4b44eb80 1 -- 172.21.15.181:0/919187379 >> 172.21.15.181:0/919187379 conn(0x55b5d01d6b20 msgr2=0x55b5d02a49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:33.948 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.946+0000 7fed4b44eb80 1 -- 172.21.15.181:0/919187379 shutdown_connections 2024-05-03T20:36:33.948 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.946+0000 7fed4b44eb80 1 -- 172.21.15.181:0/919187379 wait complete. 2024-05-03T20:36:33.948 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.946+0000 7fed4b44eb80 1 -- start start 2024-05-03T20:36:33.948 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.947+0000 7fed4b44eb80 1 Processor -- start 2024-05-03T20:36:33.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.947+0000 7fed4b44eb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02aa480 0x55b5d02b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:33.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.947+0000 7fed4b44eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5d02ae740 0x55b5d02b3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:33.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.947+0000 7fed4b44eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d02af060 0x55b5d02b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:33.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.947+0000 7fed4b44eb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b5d02b6ba0 con 0x55b5d02aa480 2024-05-03T20:36:33.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.947+0000 7fed4b44eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b5d01b5e80 con 0x55b5d02af060 2024-05-03T20:36:33.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.947+0000 7fed4b44eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b5d02b6d20 con 0x55b5d02ae740 2024-05-03T20:36:33.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.947+0000 7fed4b44eb80 10 client.0 _ll_register_callbacks cb 0x55b5d0190560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T20:36:33.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.947+0000 7fed3a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02aa480 0x55b5d02b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:33.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.947+0000 7fed3a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02aa480 0x55b5d02b5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45218/0 (socket says 172.21.15.181:45218) 2024-05-03T20:36:33.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.947+0000 7fed3a59c700 1 -- 172.21.15.181:0/2958514915 learned_addr learned my addr 172.21.15.181:0/2958514915 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:33.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.947+0000 7fed3ad9d700 1 --2- 172.21.15.181:0/2958514915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d02af060 0x55b5d02b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:33.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.947+0000 7fed31d9b700 1 --2- 172.21.15.181:0/2958514915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5d02ae740 0x55b5d02b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:33.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.947+0000 7fed3a59c700 1 -- 172.21.15.181:0/2958514915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5d02ae740 msgr2=0x55b5d02b3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:33.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.947+0000 7fed3a59c700 1 --2- 172.21.15.181:0/2958514915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5d02ae740 0x55b5d02b3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.947+0000 7fed3a59c700 1 -- 172.21.15.181:0/2958514915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d02af060 msgr2=0x55b5d02b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:33.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.947+0000 7fed3a59c700 1 --2- 172.21.15.181:0/2958514915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d02af060 0x55b5d02b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.947+0000 7fed3a59c700 1 -- 172.21.15.181:0/2958514915 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5d01b3710 con 0x55b5d02aa480 2024-05-03T20:36:33.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.947+0000 7fed31d9b700 1 --2- 172.21.15.181:0/2958514915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5d02ae740 0x55b5d02b3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:36:33.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.947+0000 7fed3a59c700 1 --2- 172.21.15.181:0/2958514915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02aa480 0x55b5d02b5e20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fed2c009990 tx=0x7fed2c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d848e051cfea07e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:33.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.947+0000 7fed21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-03T20:36:33.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.948+0000 7fed21ffb700 1 -- 172.21.15.181:0/2958514915 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2c034060 con 0x55b5d02aa480 2024-05-03T20:36:33.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.948+0000 7fed21ffb700 1 -- 172.21.15.181:0/2958514915 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7fed2c00f460 con 0x55b5d02aa480 2024-05-03T20:36:33.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.948+0000 7fed4b44eb80 1 -- 172.21.15.181:0/2958514915 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed340b1bf0 con 0x55b5d02aa480 2024-05-03T20:36:33.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.948+0000 7fed21ffb700 1 -- 172.21.15.181:0/2958514915 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fed2c00f660 con 0x55b5d02aa480 2024-05-03T20:36:33.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.948+0000 7fed4b44eb80 10 client.15078 fetch_fsmap learned FSMap version 27 2024-05-03T20:36:33.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.948+0000 7fed4b44eb80 1 -- 172.21.15.181:0/2958514915 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b5d02c13e0 con 0x55b5d02aa480 2024-05-03T20:36:33.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.948+0000 7fed21ffb700 1 -- 172.21.15.181:0/2958514915 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed340b1bf0 con 0x55b5d02aa480 2024-05-03T20:36:33.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.948+0000 7fed21ffb700 1 -- 172.21.15.181:0/2958514915 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fed2c039070 con 0x55b5d02aa480 2024-05-03T20:36:33.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.948+0000 7fed4b44eb80 10 client.15078 fetch_fsmap finished waiting for FSMap version 27 2024-05-03T20:36:33.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.948+0000 7fed4b44eb80 10 client.15078 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-03T20:36:33.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.948+0000 7fed4b44eb80 20 client.15078 populate_metadata read hostname 'smithi181' 2024-05-03T20:36:33.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.948+0000 7fed4b44eb80 1 --2- 172.21.15.181:0/2958514915 >> [v2:172.21.15.204:6834/3870850287,v1:172.21.15.204:6835/3870850287] conn(0x55b5d02c3a70 0x55b5d02c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:33.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.948+0000 7fed4b44eb80 4 client.15078 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T20:36:33.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.948+0000 7fed4b44eb80 1 -- 172.21.15.181:0/2958514915 --> [v2:172.21.15.204:6834/3870850287,v1:172.21.15.204:6835/3870850287] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5d020a2d0 con 0x55b5d02c3a70 2024-05-03T20:36:33.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.949+0000 7fed31d9b700 1 --2- 172.21.15.181:0/2958514915 >> [v2:172.21.15.204:6834/3870850287,v1:172.21.15.204:6835/3870850287] conn(0x55b5d02c3a70 0x55b5d02c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:33.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.949+0000 7fed31d9b700 1 --2- 172.21.15.181:0/2958514915 >> [v2:172.21.15.204:6834/3870850287,v1:172.21.15.204:6835/3870850287] conn(0x55b5d02c3a70 0x55b5d02c4ee0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6a801640d46b25a1 server_cookie=7094c157a9f16584 in_seq=0 out_seq=0 2024-05-03T20:36:33.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.949+0000 7fed21ffb700 10 client.15078 ms_handle_connect on v2:172.21.15.204:6834/3870850287 2024-05-03T20:36:33.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.950+0000 7fed21ffb700 1 -- 172.21.15.181:0/2958514915 <== mds.4 v2:172.21.15.204:6834/3870850287 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b5d020a2d0 con 0x55b5d02c3a70 2024-05-03T20:36:33.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.950+0000 7fed21ffb700 10 client.15078 handle_command_reply: tid=0 2024-05-03T20:36:33.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.950+0000 7fed4b44eb80 1 client.15078 shutdown 2024-05-03T20:36:33.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.950+0000 7fed4b44eb80 1 -- 172.21.15.181:0/2958514915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02aa480 msgr2=0x55b5d02b5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:33.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.950+0000 7fed4b44eb80 1 --2- 172.21.15.181:0/2958514915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02aa480 0x55b5d02b5e20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fed2c009990 tx=0x7fed2c009310 comp rx=0 tx=0).stop 2024-05-03T20:36:33.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.950+0000 7fed4b44eb80 1 -- 172.21.15.181:0/2958514915 shutdown_connections 2024-05-03T20:36:33.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.950+0000 7fed4b44eb80 1 --2- 172.21.15.181:0/2958514915 >> [v2:172.21.15.204:6834/3870850287,v1:172.21.15.204:6835/3870850287] conn(0x55b5d02c3a70 0x55b5d02c4ee0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.950+0000 7fed4b44eb80 1 --2- 172.21.15.181:0/2958514915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d02af060 0x55b5d02b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.950+0000 7fed4b44eb80 1 --2- 172.21.15.181:0/2958514915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02aa480 0x55b5d02b5e20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.950+0000 7fed4b44eb80 1 --2- 172.21.15.181:0/2958514915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5d02ae740 0x55b5d02b3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.950+0000 7fed4b44eb80 1 -- 172.21.15.181:0/2958514915 >> 172.21.15.181:0/2958514915 conn(0x55b5d01d6b20 msgr2=0x55b5d02a3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:33.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.950+0000 7fed21ffb700 0 client.15078 ms_handle_reset on v2:172.21.15.204:6834/3870850287 2024-05-03T20:36:33.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.950+0000 7fed4b44eb80 1 -- 172.21.15.181:0/2958514915 shutdown_connections 2024-05-03T20:36:33.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.950+0000 7fed4b44eb80 1 -- 172.21.15.181:0/2958514915 wait complete. 2024-05-03T20:36:33.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.950+0000 7fed4b44eb80 20 client.15078 trim_cache size 0 max 16384 2024-05-03T20:36:33.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.971+0000 7fed4b44eb80 1 Processor -- start 2024-05-03T20:36:33.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.971+0000 7fed4b44eb80 1 -- start start 2024-05-03T20:36:33.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.971+0000 7fed4b44eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d02aa480 0x55b5d02a4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:33.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.971+0000 7fed4b44eb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02ae740 0x55b5d02b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:33.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.971+0000 7fed4b44eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5d02af060 0x55b5d02b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:33.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.971+0000 7fed4b44eb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b5d02c3220 con 0x55b5d02ae740 2024-05-03T20:36:33.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.971+0000 7fed4b44eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b5d02c38f0 con 0x55b5d02aa480 2024-05-03T20:36:33.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.971+0000 7fed4b44eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b5d02c3fc0 con 0x55b5d02af060 2024-05-03T20:36:33.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.971+0000 7fed31d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02ae740 0x55b5d02b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:33.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.971+0000 7fed31d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02ae740 0x55b5d02b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45224/0 (socket says 172.21.15.181:45224) 2024-05-03T20:36:33.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.971+0000 7fed31d9b700 1 -- 172.21.15.181:0/2268120052 learned_addr learned my addr 172.21.15.181:0/2268120052 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:33.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.972+0000 7fed3a59c700 1 --2- 172.21.15.181:0/2268120052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d02aa480 0x55b5d02a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:33.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.972+0000 7fed3ad9d700 1 --2- 172.21.15.181:0/2268120052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5d02af060 0x55b5d02b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:33.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.972+0000 7fed31d9b700 1 -- 172.21.15.181:0/2268120052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5d02af060 msgr2=0x55b5d02b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:33.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.972+0000 7fed31d9b700 1 --2- 172.21.15.181:0/2268120052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5d02af060 0x55b5d02b4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.972+0000 7fed31d9b700 1 -- 172.21.15.181:0/2268120052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d02aa480 msgr2=0x55b5d02a4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:33.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.972+0000 7fed31d9b700 1 --2- 172.21.15.181:0/2268120052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d02aa480 0x55b5d02a4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.972+0000 7fed31d9b700 1 -- 172.21.15.181:0/2268120052 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5d02ae390 con 0x55b5d02ae740 2024-05-03T20:36:33.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.972+0000 7fed3a59c700 1 --2- 172.21.15.181:0/2268120052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d02aa480 0x55b5d02a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:36:33.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.972+0000 7fed3ad9d700 1 --2- 172.21.15.181:0/2268120052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5d02af060 0x55b5d02b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:36:33.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.972+0000 7fed31d9b700 1 --2- 172.21.15.181:0/2268120052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02ae740 0x55b5d02b47c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fed3c079910 tx=0x7fed2400ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ea2b38fdd279e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:33.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.972+0000 7fed23fff700 1 -- 172.21.15.181:0/2268120052 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24022070 con 0x55b5d02ae740 2024-05-03T20:36:33.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.972+0000 7fed23fff700 1 -- 172.21.15.181:0/2268120052 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed24016e20 con 0x55b5d02ae740 2024-05-03T20:36:33.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.972+0000 7fed23fff700 1 -- 172.21.15.181:0/2268120052 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed240143b0 con 0x55b5d02ae740 2024-05-03T20:36:33.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.973+0000 7fed4b44eb80 1 -- 172.21.15.181:0/2268120052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02ae740 msgr2=0x55b5d02b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:33.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.973+0000 7fed4b44eb80 1 --2- 172.21.15.181:0/2268120052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02ae740 0x55b5d02b47c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fed3c079910 tx=0x7fed2400ded0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.973+0000 7fed4b44eb80 1 -- 172.21.15.181:0/2268120052 shutdown_connections 2024-05-03T20:36:33.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.973+0000 7fed4b44eb80 1 --2- 172.21.15.181:0/2268120052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5d02af060 0x55b5d02b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.973+0000 7fed4b44eb80 1 --2- 172.21.15.181:0/2268120052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d02aa480 0x55b5d02a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.973+0000 7fed4b44eb80 1 --2- 172.21.15.181:0/2268120052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02ae740 0x55b5d02b47c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.973+0000 7fed4b44eb80 1 -- 172.21.15.181:0/2268120052 >> 172.21.15.181:0/2268120052 conn(0x55b5d01d6b20 msgr2=0x55b5d02b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:33.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.973+0000 7fed4b44eb80 1 -- 172.21.15.181:0/2268120052 shutdown_connections 2024-05-03T20:36:33.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.973+0000 7fed4b44eb80 1 -- 172.21.15.181:0/2268120052 wait complete. 2024-05-03T20:36:33.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.973+0000 7fed4b44eb80 1 -- start start 2024-05-03T20:36:33.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.973+0000 7fed4b44eb80 1 Processor -- start 2024-05-03T20:36:33.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.973+0000 7fed4b44eb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d02aa480 0x55b5d02ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:33.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.973+0000 7fed4b44eb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02ae740 0x55b5d02ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:33.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.973+0000 7fed4b44eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5d02af060 0x55b5d02c7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:33.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.973+0000 7fed4b44eb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b5d02c4d60 con 0x55b5d02ae740 2024-05-03T20:36:33.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.973+0000 7fed4b44eb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b5d02c4690 con 0x55b5d02aa480 2024-05-03T20:36:33.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.973+0000 7fed4b44eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b5d02c5430 con 0x55b5d02af060 2024-05-03T20:36:33.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.973+0000 7fed4b44eb80 10 client.0 _ll_register_callbacks cb 0x55b5d0190560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T20:36:33.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.974+0000 7fed31d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02ae740 0x55b5d02ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:33.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.974+0000 7fed31d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02ae740 0x55b5d02ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45236/0 (socket says 172.21.15.181:45236) 2024-05-03T20:36:33.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.974+0000 7fed31d9b700 1 -- 172.21.15.181:0/2340759348 learned_addr learned my addr 172.21.15.181:0/2340759348 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:33.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.974+0000 7fed3a59c700 1 --2- 172.21.15.181:0/2340759348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d02aa480 0x55b5d02ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:33.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.974+0000 7fed31d9b700 1 -- 172.21.15.181:0/2340759348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5d02af060 msgr2=0x55b5d02c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:33.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.974+0000 7fed31d9b700 1 --2- 172.21.15.181:0/2340759348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5d02af060 0x55b5d02c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.974+0000 7fed31d9b700 1 -- 172.21.15.181:0/2340759348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d02aa480 msgr2=0x55b5d02ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:33.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.974+0000 7fed31d9b700 1 --2- 172.21.15.181:0/2340759348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d02aa480 0x55b5d02ce360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.974+0000 7fed31d9b700 1 -- 172.21.15.181:0/2340759348 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5d02c7970 con 0x55b5d02ae740 2024-05-03T20:36:33.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.974+0000 7fed31d9b700 1 --2- 172.21.15.181:0/2340759348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02ae740 0x55b5d02ce8a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fed24004790 tx=0x7fed2401bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78481fb3ad4c32b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:33.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.974+0000 7fed21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-03T20:36:33.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.974+0000 7fed21ffb700 1 -- 172.21.15.181:0/2340759348 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2402c070 con 0x55b5d02ae740 2024-05-03T20:36:33.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.974+0000 7fed21ffb700 1 -- 172.21.15.181:0/2340759348 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7fed2401b3d0 con 0x55b5d02ae740 2024-05-03T20:36:33.981 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.974+0000 7fed4b44eb80 1 -- 172.21.15.181:0/2340759348 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed24016e20 con 0x55b5d02ae740 2024-05-03T20:36:33.981 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.974+0000 7fed21ffb700 1 -- 172.21.15.181:0/2340759348 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fed24038050 con 0x55b5d02ae740 2024-05-03T20:36:33.981 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.974+0000 7fed4b44eb80 10 client.15084 fetch_fsmap learned FSMap version 27 2024-05-03T20:36:33.981 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.974+0000 7fed4b44eb80 1 -- 172.21.15.181:0/2340759348 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b5d032f7b0 con 0x55b5d02ae740 2024-05-03T20:36:33.981 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.974+0000 7fed21ffb700 1 -- 172.21.15.181:0/2340759348 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed24016e20 con 0x55b5d02ae740 2024-05-03T20:36:33.981 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.975+0000 7fed21ffb700 1 -- 172.21.15.181:0/2340759348 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fed240143b0 con 0x55b5d02ae740 2024-05-03T20:36:33.981 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.975+0000 7fed4b44eb80 10 client.15084 fetch_fsmap finished waiting for FSMap version 27 2024-05-03T20:36:33.981 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.975+0000 7fed4b44eb80 10 client.15084 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-03T20:36:33.981 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.975+0000 7fed4b44eb80 20 client.15084 populate_metadata read hostname 'smithi181' 2024-05-03T20:36:33.981 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.975+0000 7fed4b44eb80 1 --2- 172.21.15.181:0/2340759348 >> [v2:172.21.15.204:6834/3870850287,v1:172.21.15.204:6835/3870850287] conn(0x55b5d0334010 0x55b5d0332d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:33.981 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.975+0000 7fed4b44eb80 4 client.15084 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T20:36:33.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.975+0000 7fed4b44eb80 1 -- 172.21.15.181:0/2340759348 --> [v2:172.21.15.204:6834/3870850287,v1:172.21.15.204:6835/3870850287] -- command(tid 0: {"prefix": "status"}) -- 0x55b5d0204370 con 0x55b5d0334010 2024-05-03T20:36:33.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.975+0000 7fed3a59c700 1 --2- 172.21.15.181:0/2340759348 >> [v2:172.21.15.204:6834/3870850287,v1:172.21.15.204:6835/3870850287] conn(0x55b5d0334010 0x55b5d0332d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:33.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.975+0000 7fed3a59c700 1 --2- 172.21.15.181:0/2340759348 >> [v2:172.21.15.204:6834/3870850287,v1:172.21.15.204:6835/3870850287] conn(0x55b5d0334010 0x55b5d0332d20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=99e9f29834c69301 server_cookie=a8d0ec6db5631a81 in_seq=0 out_seq=0 2024-05-03T20:36:33.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.975+0000 7fed21ffb700 10 client.15084 ms_handle_connect on v2:172.21.15.204:6834/3870850287 2024-05-03T20:36:33.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.976+0000 7fed21ffb700 1 -- 172.21.15.181:0/2340759348 <== mds.4 v2:172.21.15.204:6834/3870850287 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b5d0204370 con 0x55b5d0334010 2024-05-03T20:36:33.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.976+0000 7fed21ffb700 10 client.15084 handle_command_reply: tid=0 2024-05-03T20:36:33.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.976+0000 7fed4b44eb80 1 client.15084 shutdown 2024-05-03T20:36:33.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.976+0000 7fed4b44eb80 1 -- 172.21.15.181:0/2340759348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02ae740 msgr2=0x55b5d02ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:33.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.976+0000 7fed4b44eb80 1 --2- 172.21.15.181:0/2340759348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02ae740 0x55b5d02ce8a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fed24004790 tx=0x7fed2401bb50 comp rx=0 tx=0).stop 2024-05-03T20:36:33.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.976+0000 7fed4b44eb80 1 -- 172.21.15.181:0/2340759348 shutdown_connections 2024-05-03T20:36:33.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.976+0000 7fed4b44eb80 1 --2- 172.21.15.181:0/2340759348 >> [v2:172.21.15.204:6834/3870850287,v1:172.21.15.204:6835/3870850287] conn(0x55b5d0334010 0x55b5d0332d20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.976+0000 7fed4b44eb80 1 --2- 172.21.15.181:0/2340759348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b5d02af060 0x55b5d02c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.976+0000 7fed4b44eb80 1 --2- 172.21.15.181:0/2340759348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5d02aa480 0x55b5d02ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.976+0000 7fed21ffb700 0 client.15084 ms_handle_reset on v2:172.21.15.204:6834/3870850287 2024-05-03T20:36:33.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.976+0000 7fed4b44eb80 1 --2- 172.21.15.181:0/2340759348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5d02ae740 0x55b5d02ce8a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.976+0000 7fed4b44eb80 1 -- 172.21.15.181:0/2340759348 >> 172.21.15.181:0/2340759348 conn(0x55b5d01d6b20 msgr2=0x55b5d02adc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:33.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.976+0000 7fed4b44eb80 1 -- 172.21.15.181:0/2340759348 shutdown_connections 2024-05-03T20:36:33.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.976+0000 7fed4b44eb80 1 -- 172.21.15.181:0/2340759348 wait complete. 2024-05-03T20:36:33.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.976+0000 7fed4b44eb80 20 client.15084 trim_cache size 0 max 16384 2024-05-03T20:36:33.983 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-05-03T20:36:33.984 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "1cd058ee-098b-11ef-bc97-c7b262605968", 2024-05-03T20:36:33.984 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 4, 2024-05-03T20:36:33.984 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 24457, 2024-05-03T20:36:33.984 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2024-05-03T20:36:33.984 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2024-05-03T20:36:33.984 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2024-05-03T20:36:33.984 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 156.50113552400001, 2024-05-03T20:36:33.984 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 27, 2024-05-03T20:36:33.984 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 83, 2024-05-03T20:36:33.984 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 81, 2024-05-03T20:36:33.984 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 203.86354123999999 2024-05-03T20:36:33.985 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-05-03T20:36:33.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.976+0000 7fed40bbb700 1 -- 172.21.15.181:0/2002080116 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fed280b2910 msgr2=0x7fed280b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:33.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.976+0000 7fed40bbb700 1 --2- 172.21.15.181:0/2002080116 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fed280b2910 0x7fed280b4dd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00b5c0 tx=0x7fed2c005960 comp rx=0 tx=0).stop 2024-05-03T20:36:33.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.976+0000 7fed40bbb700 1 -- 172.21.15.181:0/2002080116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed3c108400 msgr2=0x7fed3c104af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:33.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.976+0000 7fed40bbb700 1 --2- 172.21.15.181:0/2002080116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed3c108400 0x7fed3c104af0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fed3400b970 tx=0x7fed3400bc80 comp rx=0 tx=0).stop 2024-05-03T20:36:33.986 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.977+0000 7fed40bbb700 1 -- 172.21.15.181:0/2002080116 shutdown_connections 2024-05-03T20:36:33.986 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.977+0000 7fed40bbb700 1 --2- 172.21.15.181:0/2002080116 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fed280b2910 0x7fed280b4dd0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.986 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.977+0000 7fed40bbb700 1 --2- 172.21.15.181:0/2002080116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed3c108400 0x7fed3c104af0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.986 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.977+0000 7fed40bbb700 1 --2- 172.21.15.181:0/2002080116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed3c102390 0x7fed3c104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.986 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.977+0000 7fed40bbb700 1 --2- 172.21.15.181:0/2002080116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed3c102d50 0x7fed3c1045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:33.986 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.977+0000 7fed40bbb700 1 -- 172.21.15.181:0/2002080116 >> 172.21.15.181:0/2002080116 conn(0x7fed3c079310 msgr2=0x7fed3c06bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:33.986 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.977+0000 7fed40bbb700 1 -- 172.21.15.181:0/2002080116 shutdown_connections 2024-05-03T20:36:33.986 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:33.977+0000 7fed40bbb700 1 -- 172.21.15.181:0/2002080116 wait complete. 2024-05-03T20:36:34.762 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-03T20:36:35.001 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:36:35.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:35 smithi181 ceph-mon[29801]: pgmap v373: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:36:35.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:35 smithi196 ceph-mon[32858]: pgmap v373: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:36:35.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:35 smithi204 ceph-mon[33595]: pgmap v373: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:36:36.565 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.561+0000 7f137719f700 1 -- 172.21.15.181:0/4035089090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1370108410 msgr2=0x7f13701087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:36.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.561+0000 7f137719f700 1 --2- 172.21.15.181:0/4035089090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1370108410 0x7f13701087f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1364009990 tx=0x7f1364009ca0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.561+0000 7f137719f700 1 -- 172.21.15.181:0/4035089090 shutdown_connections 2024-05-03T20:36:36.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.561+0000 7f137719f700 1 --2- 172.21.15.181:0/4035089090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1370102d60 0x7f137010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.561+0000 7f137719f700 1 --2- 172.21.15.181:0/4035089090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1370108410 0x7f13701087f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.561+0000 7f137719f700 1 --2- 172.21.15.181:0/4035089090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13701023a0 0x7f1370102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.561+0000 7f137719f700 1 -- 172.21.15.181:0/4035089090 >> 172.21.15.181:0/4035089090 conn(0x7f1370079350 msgr2=0x7f1370079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:36.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.561+0000 7f137719f700 1 -- 172.21.15.181:0/4035089090 shutdown_connections 2024-05-03T20:36:36.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.561+0000 7f137719f700 1 -- 172.21.15.181:0/4035089090 wait complete. 2024-05-03T20:36:36.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.562+0000 7f137719f700 1 Processor -- start 2024-05-03T20:36:36.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.562+0000 7f137719f700 1 -- start start 2024-05-03T20:36:36.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.562+0000 7f137719f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13701023a0 0x7f137019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:36.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.562+0000 7f137719f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1370102d60 0x7f137019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:36.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.562+0000 7f137719f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1370108410 0x7f13701988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:36.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.562+0000 7f137719f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1370112560 con 0x7f13701023a0 2024-05-03T20:36:36.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.562+0000 7f137719f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f13701123e0 con 0x7f1370102d60 2024-05-03T20:36:36.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.562+0000 7f137719f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f13701126e0 con 0x7f1370108410 2024-05-03T20:36:36.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.562+0000 7f1374f3b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13701023a0 0x7f137019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:36.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.562+0000 7f1374f3b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13701023a0 0x7f137019e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36366/0 (socket says 172.21.15.181:36366) 2024-05-03T20:36:36.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.562+0000 7f1374f3b700 1 -- 172.21.15.181:0/141250034 learned_addr learned my addr 172.21.15.181:0/141250034 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:36.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.562+0000 7f136ffff700 1 --2- 172.21.15.181:0/141250034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1370102d60 0x7f137019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:36.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.563+0000 7f137573c700 1 --2- 172.21.15.181:0/141250034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1370108410 0x7f13701988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:36.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.563+0000 7f1374f3b700 1 -- 172.21.15.181:0/141250034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1370108410 msgr2=0x7f13701988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:36.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.563+0000 7f1374f3b700 1 --2- 172.21.15.181:0/141250034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1370108410 0x7f13701988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.563+0000 7f1374f3b700 1 -- 172.21.15.181:0/141250034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1370102d60 msgr2=0x7f137019ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:36.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.563+0000 7f1374f3b700 1 --2- 172.21.15.181:0/141250034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1370102d60 0x7f137019ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.568 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.563+0000 7f1374f3b700 1 -- 172.21.15.181:0/141250034 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1370199020 con 0x7f13701023a0 2024-05-03T20:36:36.568 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.563+0000 7f137573c700 1 --2- 172.21.15.181:0/141250034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1370108410 0x7f13701988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:36:36.568 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.563+0000 7f136ffff700 1 --2- 172.21.15.181:0/141250034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1370102d60 0x7f137019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:36:36.568 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.563+0000 7f1374f3b700 1 --2- 172.21.15.181:0/141250034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13701023a0 0x7f137019e700 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1364000c00 tx=0x7f13640047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:36.568 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.563+0000 7f136dffb700 1 -- 172.21.15.181:0/141250034 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364027070 con 0x7f13701023a0 2024-05-03T20:36:36.568 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.563+0000 7f137719f700 1 -- 172.21.15.181:0/141250034 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13701992b0 con 0x7f13701023a0 2024-05-03T20:36:36.568 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.563+0000 7f136dffb700 1 -- 172.21.15.181:0/141250034 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13640041d0 con 0x7f13701023a0 2024-05-03T20:36:36.568 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.563+0000 7f136dffb700 1 -- 172.21.15.181:0/141250034 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136400f460 con 0x7f13701023a0 2024-05-03T20:36:36.568 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.563+0000 7f137719f700 1 -- 172.21.15.181:0/141250034 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13701a55d0 con 0x7f13701023a0 2024-05-03T20:36:36.575 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.566+0000 7f136dffb700 1 -- 172.21.15.181:0/141250034 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f136402a070 con 0x7f13701023a0 2024-05-03T20:36:36.575 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.566+0000 7f136dffb700 1 --2- 172.21.15.181:0/141250034 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f13600b2910 0x7f13600b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:36.575 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.566+0000 7f136ffff700 1 --2- 172.21.15.181:0/141250034 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f13600b2910 0x7f13600b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:36.575 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.566+0000 7f136dffb700 1 -- 172.21.15.181:0/141250034 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f13640e3340 con 0x7f13701023a0 2024-05-03T20:36:36.575 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.567+0000 7f136ffff700 1 --2- 172.21.15.181:0/141250034 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f13600b2910 0x7f13600b4dd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f13700799a0 tx=0x7f135c005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:36.575 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.567+0000 7f1381a32b80 1 Processor -- start 2024-05-03T20:36:36.575 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.567+0000 7f1381a32b80 1 -- start start 2024-05-03T20:36:36.575 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.567+0000 7f1381a32b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed674e150 0x563ed674e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:36.575 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.567+0000 7f1381a32b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ed674ca30 0x563ed674ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:36.575 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.567+0000 7f1381a32b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563ed674d3f0 0x563ed675d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:36.575 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.567+0000 7f1381a32b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563ed669f810 con 0x563ed674e150 2024-05-03T20:36:36.575 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.567+0000 7f1381a32b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563ed663e770 con 0x563ed674ca30 2024-05-03T20:36:36.575 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.567+0000 7f1381a32b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563ed669b7d0 con 0x563ed674d3f0 2024-05-03T20:36:36.576 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.567+0000 7f1374f3b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed674e150 0x563ed674e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:36.576 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.567+0000 7f1374f3b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed674e150 0x563ed674e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36382/0 (socket says 172.21.15.181:36382) 2024-05-03T20:36:36.576 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.567+0000 7f1374f3b700 1 -- 172.21.15.181:0/3857888274 learned_addr learned my addr 172.21.15.181:0/3857888274 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:36.576 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.567+0000 7f136ffff700 1 --2- 172.21.15.181:0/3857888274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ed674ca30 0x563ed674ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:36.576 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.567+0000 7f137573c700 1 --2- 172.21.15.181:0/3857888274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563ed674d3f0 0x563ed675d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:36.576 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.567+0000 7f1374f3b700 1 -- 172.21.15.181:0/3857888274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563ed674d3f0 msgr2=0x563ed675d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:36.576 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.567+0000 7f1374f3b700 1 --2- 172.21.15.181:0/3857888274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563ed674d3f0 0x563ed675d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.576 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.567+0000 7f1374f3b700 1 -- 172.21.15.181:0/3857888274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ed674ca30 msgr2=0x563ed674ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:36.576 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.567+0000 7f1374f3b700 1 --2- 172.21.15.181:0/3857888274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ed674ca30 0x563ed674ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.576 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.567+0000 7f1374f3b700 1 -- 172.21.15.181:0/3857888274 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ed65ec880 con 0x563ed674e150 2024-05-03T20:36:36.576 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.567+0000 7f136ffff700 1 --2- 172.21.15.181:0/3857888274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ed674ca30 0x563ed674ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:36:36.576 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.567+0000 7f137573c700 1 --2- 172.21.15.181:0/3857888274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563ed674d3f0 0x563ed675d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:36:36.576 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.568+0000 7f1374f3b700 1 --2- 172.21.15.181:0/3857888274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed674e150 0x563ed674e530 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1364003800 tx=0x7f13640aa000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970a74a982badbf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:36.576 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.568+0000 7f135affd700 1 -- 172.21.15.181:0/3857888274 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364022470 con 0x563ed674e150 2024-05-03T20:36:36.577 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.568+0000 7f135affd700 1 -- 172.21.15.181:0/3857888274 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13640aa890 con 0x563ed674e150 2024-05-03T20:36:36.577 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.568+0000 7f135affd700 1 -- 172.21.15.181:0/3857888274 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13640aaaf0 con 0x563ed674e150 2024-05-03T20:36:36.577 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.568+0000 7f1381a32b80 1 -- 172.21.15.181:0/3857888274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed674e150 msgr2=0x563ed674e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:36.577 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.568+0000 7f1381a32b80 1 --2- 172.21.15.181:0/3857888274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed674e150 0x563ed674e530 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1364003800 tx=0x7f13640aa000 comp rx=0 tx=0).stop 2024-05-03T20:36:36.577 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.569+0000 7f1381a32b80 1 -- 172.21.15.181:0/3857888274 shutdown_connections 2024-05-03T20:36:36.577 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.569+0000 7f1381a32b80 1 --2- 172.21.15.181:0/3857888274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563ed674d3f0 0x563ed675d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.577 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.569+0000 7f1381a32b80 1 --2- 172.21.15.181:0/3857888274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed674e150 0x563ed674e530 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.577 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.569+0000 7f1381a32b80 1 --2- 172.21.15.181:0/3857888274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ed674ca30 0x563ed674ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.577 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.569+0000 7f1381a32b80 1 -- 172.21.15.181:0/3857888274 >> 172.21.15.181:0/3857888274 conn(0x563ed66790b0 msgr2=0x563ed6747280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:36.577 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.569+0000 7f1381a32b80 1 -- 172.21.15.181:0/3857888274 shutdown_connections 2024-05-03T20:36:36.577 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.569+0000 7f1381a32b80 1 -- 172.21.15.181:0/3857888274 wait complete. 2024-05-03T20:36:36.577 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.569+0000 7f1381a32b80 1 -- start start 2024-05-03T20:36:36.577 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.569+0000 7f1381a32b80 1 Processor -- start 2024-05-03T20:36:36.577 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.569+0000 7f1381a32b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed674ca30 0x563ed675cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:36.577 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.569+0000 7f1381a32b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ed674d3f0 0x563ed675a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:36.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.569+0000 7f1381a32b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563ed674e150 0x563ed675a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:36.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.569+0000 7f1381a32b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563ed675d6c0 con 0x563ed674ca30 2024-05-03T20:36:36.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.569+0000 7f1381a32b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563ed6605be0 con 0x563ed674d3f0 2024-05-03T20:36:36.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.569+0000 7f1381a32b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563ed675d840 con 0x563ed674e150 2024-05-03T20:36:36.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.570+0000 7f1381a32b80 10 client.0 _ll_register_callbacks cb 0x563ed66378d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T20:36:36.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.570+0000 7f1374f3b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed674ca30 0x563ed675cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:36.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.570+0000 7f1374f3b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed674ca30 0x563ed675cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36390/0 (socket says 172.21.15.181:36390) 2024-05-03T20:36:36.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.570+0000 7f1374f3b700 1 -- 172.21.15.181:0/2150478674 learned_addr learned my addr 172.21.15.181:0/2150478674 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:36.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.570+0000 7f136ffff700 1 --2- 172.21.15.181:0/2150478674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ed674d3f0 0x563ed675a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:36.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.570+0000 7f1374f3b700 1 -- 172.21.15.181:0/2150478674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563ed674e150 msgr2=0x563ed675a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:36.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.570+0000 7f1374f3b700 1 --2- 172.21.15.181:0/2150478674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563ed674e150 0x563ed675a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.570+0000 7f1374f3b700 1 -- 172.21.15.181:0/2150478674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ed674d3f0 msgr2=0x563ed675a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:36.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.570+0000 7f1374f3b700 1 --2- 172.21.15.181:0/2150478674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ed674d3f0 0x563ed675a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.570+0000 7f1374f3b700 1 -- 172.21.15.181:0/2150478674 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ed663b640 con 0x563ed674ca30 2024-05-03T20:36:36.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.570+0000 7f136ffff700 1 --2- 172.21.15.181:0/2150478674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ed674d3f0 0x563ed675a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:36:36.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.570+0000 7f1374f3b700 1 --2- 172.21.15.181:0/2150478674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed674ca30 0x563ed675cd10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f13640a6910 tx=0x7f13640b3660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61146ec9849cdbd server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:36.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.570+0000 7f134ffff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-03T20:36:36.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.570+0000 7f134ffff700 1 -- 172.21.15.181:0/2150478674 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13640b10b0 con 0x563ed674ca30 2024-05-03T20:36:36.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.570+0000 7f134ffff700 1 -- 172.21.15.181:0/2150478674 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f13640b31d0 con 0x563ed674ca30 2024-05-03T20:36:36.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.570+0000 7f1381a32b80 1 -- 172.21.15.181:0/2150478674 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13640aa890 con 0x563ed674ca30 2024-05-03T20:36:36.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.570+0000 7f134ffff700 1 -- 172.21.15.181:0/2150478674 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f13640b33d0 con 0x563ed674ca30 2024-05-03T20:36:36.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.570+0000 7f1381a32b80 10 client.15096 fetch_fsmap learned FSMap version 27 2024-05-03T20:36:36.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.570+0000 7f1381a32b80 1 -- 172.21.15.181:0/2150478674 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563ed675d6c0 con 0x563ed674ca30 2024-05-03T20:36:36.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.570+0000 7f134ffff700 1 -- 172.21.15.181:0/2150478674 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13640aa890 con 0x563ed674ca30 2024-05-03T20:36:36.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.571+0000 7f134ffff700 1 -- 172.21.15.181:0/2150478674 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f13640a6200 con 0x563ed674ca30 2024-05-03T20:36:36.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.571+0000 7f1381a32b80 10 client.15096 fetch_fsmap finished waiting for FSMap version 27 2024-05-03T20:36:36.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.571+0000 7f1381a32b80 10 client.15096 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-03T20:36:36.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.571+0000 7f1381a32b80 20 client.15096 populate_metadata read hostname 'smithi181' 2024-05-03T20:36:36.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.571+0000 7f1381a32b80 1 --2- 172.21.15.181:0/2150478674 >> [v2:172.21.15.204:6836/3006728559,v1:172.21.15.204:6837/3006728559] conn(0x563ed676a210 0x563ed676b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:36.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.571+0000 7f1381a32b80 4 client.15096 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T20:36:36.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.571+0000 7f1381a32b80 1 -- 172.21.15.181:0/2150478674 --> [v2:172.21.15.204:6836/3006728559,v1:172.21.15.204:6837/3006728559] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ed66ad1f0 con 0x563ed676a210 2024-05-03T20:36:36.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.571+0000 7f136ffff700 1 --2- 172.21.15.181:0/2150478674 >> [v2:172.21.15.204:6836/3006728559,v1:172.21.15.204:6837/3006728559] conn(0x563ed676a210 0x563ed676b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:36.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.571+0000 7f136ffff700 1 --2- 172.21.15.181:0/2150478674 >> [v2:172.21.15.204:6836/3006728559,v1:172.21.15.204:6837/3006728559] conn(0x563ed676a210 0x563ed676b680 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b0679e91041acb0d server_cookie=68d466ca7e0109cf in_seq=0 out_seq=0 2024-05-03T20:36:36.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.571+0000 7f134ffff700 10 client.15096 ms_handle_connect on v2:172.21.15.204:6836/3006728559 2024-05-03T20:36:36.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.572+0000 7f134ffff700 1 -- 172.21.15.181:0/2150478674 <== mds.3 v2:172.21.15.204:6836/3006728559 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563ed66ad1f0 con 0x563ed676a210 2024-05-03T20:36:36.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.572+0000 7f134ffff700 10 client.15096 handle_command_reply: tid=0 2024-05-03T20:36:36.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.572+0000 7f1381a32b80 1 client.15096 shutdown 2024-05-03T20:36:36.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.572+0000 7f1381a32b80 1 -- 172.21.15.181:0/2150478674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed674ca30 msgr2=0x563ed675cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:36.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.572+0000 7f1381a32b80 1 --2- 172.21.15.181:0/2150478674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed674ca30 0x563ed675cd10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f13640a6910 tx=0x7f13640b3660 comp rx=0 tx=0).stop 2024-05-03T20:36:36.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.572+0000 7f1381a32b80 1 -- 172.21.15.181:0/2150478674 shutdown_connections 2024-05-03T20:36:36.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.572+0000 7f1381a32b80 1 --2- 172.21.15.181:0/2150478674 >> [v2:172.21.15.204:6836/3006728559,v1:172.21.15.204:6837/3006728559] conn(0x563ed676a210 0x563ed676b680 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.572+0000 7f1381a32b80 1 --2- 172.21.15.181:0/2150478674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563ed674e150 0x563ed675a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.572+0000 7f1381a32b80 1 --2- 172.21.15.181:0/2150478674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed674ca30 0x563ed675cd10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.572+0000 7f1381a32b80 1 --2- 172.21.15.181:0/2150478674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ed674d3f0 0x563ed675a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.572+0000 7f1381a32b80 1 -- 172.21.15.181:0/2150478674 >> 172.21.15.181:0/2150478674 conn(0x563ed66790b0 msgr2=0x563ed67468f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:36.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.572+0000 7f134ffff700 0 client.15096 ms_handle_reset on v2:172.21.15.204:6836/3006728559 2024-05-03T20:36:36.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.573+0000 7f1381a32b80 1 -- 172.21.15.181:0/2150478674 shutdown_connections 2024-05-03T20:36:36.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.573+0000 7f1381a32b80 1 -- 172.21.15.181:0/2150478674 wait complete. 2024-05-03T20:36:36.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.573+0000 7f1381a32b80 20 client.15096 trim_cache size 0 max 16384 2024-05-03T20:36:36.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.593+0000 7f1381a32b80 1 Processor -- start 2024-05-03T20:36:36.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.593+0000 7f1381a32b80 1 -- start start 2024-05-03T20:36:36.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.593+0000 7f1381a32b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed66790b0 0x563ed67558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:36.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.593+0000 7f1381a32b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ed674ca30 0x563ed6759010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:36.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.593+0000 7f1381a32b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563ed674d3f0 0x563ed674c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:36.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.593+0000 7f1381a32b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563ed67699c0 con 0x563ed66790b0 2024-05-03T20:36:36.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.593+0000 7f1381a32b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563ed676a090 con 0x563ed674ca30 2024-05-03T20:36:36.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.593+0000 7f1381a32b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563ed676a760 con 0x563ed674d3f0 2024-05-03T20:36:36.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.593+0000 7f1374f3b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed66790b0 0x563ed67558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:36.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.593+0000 7f1374f3b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed66790b0 0x563ed67558f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36404/0 (socket says 172.21.15.181:36404) 2024-05-03T20:36:36.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.593+0000 7f1374f3b700 1 -- 172.21.15.181:0/1638193545 learned_addr learned my addr 172.21.15.181:0/1638193545 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:36.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.594+0000 7f137573c700 1 --2- 172.21.15.181:0/1638193545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563ed674d3f0 0x563ed674c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:36.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.594+0000 7f136ffff700 1 --2- 172.21.15.181:0/1638193545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ed674ca30 0x563ed6759010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:36.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.594+0000 7f1374f3b700 1 -- 172.21.15.181:0/1638193545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563ed674d3f0 msgr2=0x563ed674c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:36.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.594+0000 7f1374f3b700 1 --2- 172.21.15.181:0/1638193545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563ed674d3f0 0x563ed674c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.594+0000 7f1374f3b700 1 -- 172.21.15.181:0/1638193545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ed674ca30 msgr2=0x563ed6759010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:36.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.594+0000 7f1374f3b700 1 --2- 172.21.15.181:0/1638193545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ed674ca30 0x563ed6759010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.594+0000 7f1374f3b700 1 -- 172.21.15.181:0/1638193545 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ed6758620 con 0x563ed66790b0 2024-05-03T20:36:36.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.594+0000 7f137573c700 1 --2- 172.21.15.181:0/1638193545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563ed674d3f0 0x563ed674c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:36:36.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.594+0000 7f136ffff700 1 --2- 172.21.15.181:0/1638193545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ed674ca30 0x563ed6759010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:36:36.600 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.594+0000 7f1374f3b700 1 --2- 172.21.15.181:0/1638193545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed66790b0 0x563ed67558f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f13640043a0 tx=0x7f13640aa000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c22b2a1c39bebe server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:36.600 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.594+0000 7f135a7fc700 1 -- 172.21.15.181:0/1638193545 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13640aa890 con 0x563ed66790b0 2024-05-03T20:36:36.600 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.594+0000 7f135a7fc700 1 -- 172.21.15.181:0/1638193545 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13640a9af0 con 0x563ed66790b0 2024-05-03T20:36:36.600 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.594+0000 7f135a7fc700 1 -- 172.21.15.181:0/1638193545 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13640a6200 con 0x563ed66790b0 2024-05-03T20:36:36.600 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.595+0000 7f1381a32b80 1 -- 172.21.15.181:0/1638193545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed66790b0 msgr2=0x563ed67558f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:36.600 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.595+0000 7f1381a32b80 1 --2- 172.21.15.181:0/1638193545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed66790b0 0x563ed67558f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f13640043a0 tx=0x7f13640aa000 comp rx=0 tx=0).stop 2024-05-03T20:36:36.600 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.595+0000 7f1381a32b80 1 -- 172.21.15.181:0/1638193545 shutdown_connections 2024-05-03T20:36:36.600 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.595+0000 7f1381a32b80 1 --2- 172.21.15.181:0/1638193545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563ed674d3f0 0x563ed674c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.595+0000 7f1381a32b80 1 --2- 172.21.15.181:0/1638193545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed66790b0 0x563ed67558f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.595+0000 7f1381a32b80 1 --2- 172.21.15.181:0/1638193545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ed674ca30 0x563ed6759010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.595+0000 7f1381a32b80 1 -- 172.21.15.181:0/1638193545 >> 172.21.15.181:0/1638193545 conn(0x563ed675afa0 msgr2=0x563ed674ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:36.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.595+0000 7f1381a32b80 1 -- 172.21.15.181:0/1638193545 shutdown_connections 2024-05-03T20:36:36.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.595+0000 7f1381a32b80 1 -- 172.21.15.181:0/1638193545 wait complete. 2024-05-03T20:36:36.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.595+0000 7f1381a32b80 1 -- start start 2024-05-03T20:36:36.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.596+0000 7f1381a32b80 1 Processor -- start 2024-05-03T20:36:36.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.596+0000 7f1381a32b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563ed674ca30 0x563ed67750e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:36.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.596+0000 7f1381a32b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ed674d3f0 0x563ed676e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:36.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.596+0000 7f1381a32b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed674e150 0x563ed676e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:36.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.596+0000 7f1381a32b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563ed676b500 con 0x563ed674e150 2024-05-03T20:36:36.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.596+0000 7f1381a32b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563ed676ae30 con 0x563ed674d3f0 2024-05-03T20:36:36.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.596+0000 7f1381a32b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563ed676bbd0 con 0x563ed674ca30 2024-05-03T20:36:36.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.596+0000 7f1381a32b80 10 client.0 _ll_register_callbacks cb 0x563ed66378d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T20:36:36.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.596+0000 7f137573c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed674e150 0x563ed676e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:36.602 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.596+0000 7f137573c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed674e150 0x563ed676e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36418/0 (socket says 172.21.15.181:36418) 2024-05-03T20:36:36.602 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.596+0000 7f137573c700 1 -- 172.21.15.181:0/1308938655 learned_addr learned my addr 172.21.15.181:0/1308938655 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:36.602 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.596+0000 7f1374f3b700 1 --2- 172.21.15.181:0/1308938655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563ed674ca30 0x563ed67750e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:36.602 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.596+0000 7f136ffff700 1 --2- 172.21.15.181:0/1308938655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ed674d3f0 0x563ed676e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:36.602 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.596+0000 7f137573c700 1 -- 172.21.15.181:0/1308938655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563ed674ca30 msgr2=0x563ed67750e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:36.602 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.596+0000 7f137573c700 1 --2- 172.21.15.181:0/1308938655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563ed674ca30 0x563ed67750e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.603 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.596+0000 7f137573c700 1 -- 172.21.15.181:0/1308938655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ed674d3f0 msgr2=0x563ed676e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:36.603 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.596+0000 7f137573c700 1 --2- 172.21.15.181:0/1308938655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ed674d3f0 0x563ed676e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.603 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.597+0000 7f137573c700 1 -- 172.21.15.181:0/1308938655 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ed676f1a0 con 0x563ed674e150 2024-05-03T20:36:36.603 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.597+0000 7f137573c700 1 --2- 172.21.15.181:0/1308938655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed674e150 0x563ed676e9d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1368008a20 tx=0x7f1368008d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9102dcd46213784 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:36.603 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.597+0000 7f134ffff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-03T20:36:36.603 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.597+0000 7f134ffff700 1 -- 172.21.15.181:0/1308938655 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1368018020 con 0x563ed674e150 2024-05-03T20:36:36.603 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.597+0000 7f134ffff700 1 -- 172.21.15.181:0/1308938655 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f136800fac0 con 0x563ed674e150 2024-05-03T20:36:36.603 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.597+0000 7f1381a32b80 1 -- 172.21.15.181:0/1308938655 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13640a9af0 con 0x563ed674e150 2024-05-03T20:36:36.603 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.597+0000 7f134ffff700 1 -- 172.21.15.181:0/1308938655 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f136800fcc0 con 0x563ed674e150 2024-05-03T20:36:36.603 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.597+0000 7f1381a32b80 10 client.15102 fetch_fsmap learned FSMap version 27 2024-05-03T20:36:36.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.597+0000 7f1381a32b80 1 -- 172.21.15.181:0/1308938655 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563ed67d5740 con 0x563ed674e150 2024-05-03T20:36:36.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.597+0000 7f134ffff700 1 -- 172.21.15.181:0/1308938655 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13640a9af0 con 0x563ed674e150 2024-05-03T20:36:36.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.597+0000 7f134ffff700 1 -- 172.21.15.181:0/1308938655 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f13680133c0 con 0x563ed674e150 2024-05-03T20:36:36.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.597+0000 7f1381a32b80 10 client.15102 fetch_fsmap finished waiting for FSMap version 27 2024-05-03T20:36:36.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.598+0000 7f1381a32b80 10 client.15102 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-03T20:36:36.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.598+0000 7f1381a32b80 20 client.15102 populate_metadata read hostname 'smithi181' 2024-05-03T20:36:36.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.598+0000 7f1381a32b80 1 --2- 172.21.15.181:0/1308938655 >> [v2:172.21.15.204:6836/3006728559,v1:172.21.15.204:6837/3006728559] conn(0x563ed67da340 0x563ed67d9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:36.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.598+0000 7f1381a32b80 4 client.15102 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T20:36:36.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.598+0000 7f1381a32b80 1 -- 172.21.15.181:0/1308938655 --> [v2:172.21.15.204:6836/3006728559,v1:172.21.15.204:6837/3006728559] -- command(tid 0: {"prefix": "status"}) -- 0x563ed66a7290 con 0x563ed67da340 2024-05-03T20:36:36.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.598+0000 7f1374f3b700 1 --2- 172.21.15.181:0/1308938655 >> [v2:172.21.15.204:6836/3006728559,v1:172.21.15.204:6837/3006728559] conn(0x563ed67da340 0x563ed67d9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:36.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.598+0000 7f1374f3b700 1 --2- 172.21.15.181:0/1308938655 >> [v2:172.21.15.204:6836/3006728559,v1:172.21.15.204:6837/3006728559] conn(0x563ed67da340 0x563ed67d9050 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fb075c1dd447ce4d server_cookie=bcdd9a13fde994b3 in_seq=0 out_seq=0 2024-05-03T20:36:36.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.598+0000 7f134ffff700 10 client.15102 ms_handle_connect on v2:172.21.15.204:6836/3006728559 2024-05-03T20:36:36.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.598+0000 7f134ffff700 1 -- 172.21.15.181:0/1308938655 <== mds.3 v2:172.21.15.204:6836/3006728559 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x563ed66a7290 con 0x563ed67da340 2024-05-03T20:36:36.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.598+0000 7f134ffff700 10 client.15102 handle_command_reply: tid=0 2024-05-03T20:36:36.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.598+0000 7f1381a32b80 1 client.15102 shutdown 2024-05-03T20:36:36.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.599+0000 7f1381a32b80 1 -- 172.21.15.181:0/1308938655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed674e150 msgr2=0x563ed676e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:36.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.599+0000 7f1381a32b80 1 --2- 172.21.15.181:0/1308938655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed674e150 0x563ed676e9d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1368008a20 tx=0x7f1368008d30 comp rx=0 tx=0).stop 2024-05-03T20:36:36.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.599+0000 7f1381a32b80 1 -- 172.21.15.181:0/1308938655 shutdown_connections 2024-05-03T20:36:36.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.599+0000 7f1381a32b80 1 --2- 172.21.15.181:0/1308938655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563ed674e150 0x563ed676e9d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.599+0000 7f1381a32b80 1 --2- 172.21.15.181:0/1308938655 >> [v2:172.21.15.204:6836/3006728559,v1:172.21.15.204:6837/3006728559] conn(0x563ed67da340 0x563ed67d9050 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.599+0000 7f1381a32b80 1 --2- 172.21.15.181:0/1308938655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563ed674ca30 0x563ed67750e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.599+0000 7f134ffff700 0 client.15102 ms_handle_reset on v2:172.21.15.204:6836/3006728559 2024-05-03T20:36:36.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.599+0000 7f1381a32b80 1 --2- 172.21.15.181:0/1308938655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563ed674d3f0 0x563ed676e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.599+0000 7f1381a32b80 1 -- 172.21.15.181:0/1308938655 >> 172.21.15.181:0/1308938655 conn(0x563ed675afa0 msgr2=0x563ed6755770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:36.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.599+0000 7f1381a32b80 1 -- 172.21.15.181:0/1308938655 shutdown_connections 2024-05-03T20:36:36.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.599+0000 7f1381a32b80 1 -- 172.21.15.181:0/1308938655 wait complete. 2024-05-03T20:36:36.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.599+0000 7f1381a32b80 20 client.15102 trim_cache size 0 max 16384 2024-05-03T20:36:36.606 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-05-03T20:36:36.606 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "1cd058ee-098b-11ef-bc97-c7b262605968", 2024-05-03T20:36:36.606 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 3, 2024-05-03T20:36:36.606 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 24463, 2024-05-03T20:36:36.606 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2024-05-03T20:36:36.606 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2024-05-03T20:36:36.606 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2024-05-03T20:36:36.606 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 160.13268843200001, 2024-05-03T20:36:36.606 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 27, 2024-05-03T20:36:36.607 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 83, 2024-05-03T20:36:36.607 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 81, 2024-05-03T20:36:36.607 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 203.72118921399999 2024-05-03T20:36:36.607 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-05-03T20:36:36.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.600+0000 7f137719f700 1 -- 172.21.15.181:0/141250034 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f13600b2910 msgr2=0x7f13600b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:36.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.600+0000 7f137719f700 1 --2- 172.21.15.181:0/141250034 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f13600b2910 0x7f13600b4dd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f13700799a0 tx=0x7f135c005f30 comp rx=0 tx=0).stop 2024-05-03T20:36:36.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.600+0000 7f137719f700 1 -- 172.21.15.181:0/141250034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13701023a0 msgr2=0x7f137019e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:36.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.600+0000 7f137719f700 1 --2- 172.21.15.181:0/141250034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13701023a0 0x7f137019e700 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1364000c00 tx=0x7f13640047b0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.600+0000 7f137719f700 1 -- 172.21.15.181:0/141250034 shutdown_connections 2024-05-03T20:36:36.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.600+0000 7f137719f700 1 --2- 172.21.15.181:0/141250034 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f13600b2910 0x7f13600b4dd0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.600+0000 7f137719f700 1 --2- 172.21.15.181:0/141250034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1370108410 0x7f13701988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.600+0000 7f137719f700 1 --2- 172.21.15.181:0/141250034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13701023a0 0x7f137019e700 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.600+0000 7f137719f700 1 --2- 172.21.15.181:0/141250034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1370102d60 0x7f137019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:36.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.600+0000 7f137719f700 1 -- 172.21.15.181:0/141250034 >> 172.21.15.181:0/141250034 conn(0x7f1370079350 msgr2=0x7f13701011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:36.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.600+0000 7f137719f700 1 -- 172.21.15.181:0/141250034 shutdown_connections 2024-05-03T20:36:36.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:36.600+0000 7f137719f700 1 -- 172.21.15.181:0/141250034 wait complete. 2024-05-03T20:36:37.477 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph tell mds.h status 2024-05-03T20:36:37.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:37 smithi181 ceph-mon[29801]: pgmap v374: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:36:37.720 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:36:37.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:37 smithi196 ceph-mon[32858]: pgmap v374: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:36:37.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:37 smithi204 ceph-mon[33595]: pgmap v374: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:36:39.349 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.349+0000 7f3ba8459700 1 -- 172.21.15.181:0/1810515284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ba0103750 msgr2=0x7f3ba0103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:39.349 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.349+0000 7f3ba8459700 1 --2- 172.21.15.181:0/1810515284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ba0103750 0x7f3ba0103b30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800b0d0 tx=0x7f3b9800b3e0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.349 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.349+0000 7f3ba8459700 1 -- 172.21.15.181:0/1810515284 shutdown_connections 2024-05-03T20:36:39.349 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.349+0000 7f3ba8459700 1 --2- 172.21.15.181:0/1810515284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ba0108850 0x7f3ba01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.349 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.349+0000 7f3ba8459700 1 --2- 172.21.15.181:0/1810515284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ba0103750 0x7f3ba0103b30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.349 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.349+0000 7f3ba8459700 1 --2- 172.21.15.181:0/1810515284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ba0104100 0x7f3ba0108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.349+0000 7f3ba8459700 1 -- 172.21.15.181:0/1810515284 >> 172.21.15.181:0/1810515284 conn(0x7f3ba007a600 msgr2=0x7f3ba007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:39.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.349+0000 7f3ba8459700 1 -- 172.21.15.181:0/1810515284 shutdown_connections 2024-05-03T20:36:39.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.349+0000 7f3ba8459700 1 -- 172.21.15.181:0/1810515284 wait complete. 2024-05-03T20:36:39.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.349+0000 7f3ba8459700 1 Processor -- start 2024-05-03T20:36:39.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.349+0000 7f3ba8459700 1 -- start start 2024-05-03T20:36:39.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.350+0000 7f3ba8459700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ba0103750 0x7f3ba019f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:39.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.350+0000 7f3ba8459700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ba0104100 0x7f3ba019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:39.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.350+0000 7f3ba8459700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ba0108850 0x7f3ba01999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:39.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.350+0000 7f3ba8459700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba01136a0 con 0x7f3ba0104100 2024-05-03T20:36:39.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.350+0000 7f3ba8459700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba0113520 con 0x7f3ba0103750 2024-05-03T20:36:39.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.350+0000 7f3ba8459700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba0113820 con 0x7f3ba0108850 2024-05-03T20:36:39.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.350+0000 7f3ba59f4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ba0104100 0x7f3ba019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:39.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.350+0000 7f3ba59f4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ba0104100 0x7f3ba019fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36436/0 (socket says 172.21.15.181:36436) 2024-05-03T20:36:39.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.350+0000 7f3ba59f4700 1 -- 172.21.15.181:0/3758626319 learned_addr learned my addr 172.21.15.181:0/3758626319 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:39.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.350+0000 7f3ba61f5700 1 --2- 172.21.15.181:0/3758626319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ba0103750 0x7f3ba019f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:39.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.350+0000 7f3ba59f4700 1 -- 172.21.15.181:0/3758626319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ba0108850 msgr2=0x7f3ba01999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:39.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.350+0000 7f3ba59f4700 1 --2- 172.21.15.181:0/3758626319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ba0108850 0x7f3ba01999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.350+0000 7f3ba59f4700 1 -- 172.21.15.181:0/3758626319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ba0103750 msgr2=0x7f3ba019f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:39.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.350+0000 7f3ba59f4700 1 --2- 172.21.15.181:0/3758626319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ba0103750 0x7f3ba019f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.350+0000 7f3ba59f4700 1 -- 172.21.15.181:0/3758626319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba019a120 con 0x7f3ba0104100 2024-05-03T20:36:39.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.350+0000 7f3ba59f4700 1 --2- 172.21.15.181:0/3758626319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ba0104100 0x7f3ba019fd40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3b9400c920 tx=0x7f3b9400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:39.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.350+0000 7f3b9f7fe700 1 -- 172.21.15.181:0/3758626319 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b94007b00 con 0x7f3ba0104100 2024-05-03T20:36:39.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.350+0000 7f3b9f7fe700 1 -- 172.21.15.181:0/3758626319 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b94004ce0 con 0x7f3ba0104100 2024-05-03T20:36:39.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.350+0000 7f3ba8459700 1 -- 172.21.15.181:0/3758626319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ba019a410 con 0x7f3ba0104100 2024-05-03T20:36:39.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.350+0000 7f3b9f7fe700 1 -- 172.21.15.181:0/3758626319 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b94005650 con 0x7f3ba0104100 2024-05-03T20:36:39.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.350+0000 7f3ba8459700 1 -- 172.21.15.181:0/3758626319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ba01a66d0 con 0x7f3ba0104100 2024-05-03T20:36:39.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.353+0000 7f3b9f7fe700 1 -- 172.21.15.181:0/3758626319 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f3b94004750 con 0x7f3ba0104100 2024-05-03T20:36:39.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.353+0000 7f3b9f7fe700 1 --2- 172.21.15.181:0/3758626319 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3b880b2910 0x7f3b880b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:39.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.353+0000 7f3ba61f5700 1 --2- 172.21.15.181:0/3758626319 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3b880b2910 0x7f3b880b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:39.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.353+0000 7f3b9f7fe700 1 -- 172.21.15.181:0/3758626319 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f3b940e1fe0 con 0x7f3ba0104100 2024-05-03T20:36:39.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.354+0000 7f3ba61f5700 1 --2- 172.21.15.181:0/3758626319 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3b880b2910 0x7f3b880b4dd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800d010 tx=0x7f3b9800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:39.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.354+0000 7f3bb2cecb80 1 Processor -- start 2024-05-03T20:36:39.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.354+0000 7f3bb2cecb80 1 -- start start 2024-05-03T20:36:39.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.354+0000 7f3bb2cecb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e42d00 0x559a19e430e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:39.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.354+0000 7f3bb2cecb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a19e43620 0x559a19e3d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:39.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.354+0000 7f3bb2cecb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a19e3d8e0 0x559a19e3dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:39.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.354+0000 7f3bb2cecb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559a19d8b8f0 con 0x559a19e42d00 2024-05-03T20:36:39.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.354+0000 7f3bb2cecb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559a19cf0d50 con 0x559a19e43620 2024-05-03T20:36:39.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.354+0000 7f3bb2cecb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559a19d878b0 con 0x559a19e3d8e0 2024-05-03T20:36:39.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.354+0000 7f3ba61f5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e42d00 0x559a19e430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:39.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.354+0000 7f3ba61f5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e42d00 0x559a19e430e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36448/0 (socket says 172.21.15.181:36448) 2024-05-03T20:36:39.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.354+0000 7f3ba61f5700 1 -- 172.21.15.181:0/3112109620 learned_addr learned my addr 172.21.15.181:0/3112109620 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:39.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.355+0000 7f3ba59f4700 1 --2- 172.21.15.181:0/3112109620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a19e43620 0x559a19e3d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:39.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.355+0000 7f3ba69f6700 1 --2- 172.21.15.181:0/3112109620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a19e3d8e0 0x559a19e3dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:39.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.355+0000 7f3ba61f5700 1 -- 172.21.15.181:0/3112109620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a19e3d8e0 msgr2=0x559a19e3dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:39.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.355+0000 7f3ba61f5700 1 --2- 172.21.15.181:0/3112109620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a19e3d8e0 0x559a19e3dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.355+0000 7f3ba61f5700 1 -- 172.21.15.181:0/3112109620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a19e43620 msgr2=0x559a19e3d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:39.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.355+0000 7f3ba61f5700 1 --2- 172.21.15.181:0/3112109620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a19e43620 0x559a19e3d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.355+0000 7f3ba61f5700 1 -- 172.21.15.181:0/3112109620 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a19cd8880 con 0x559a19e42d00 2024-05-03T20:36:39.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.355+0000 7f3ba69f6700 1 --2- 172.21.15.181:0/3112109620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a19e3d8e0 0x559a19e3dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:36:39.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.355+0000 7f3ba61f5700 1 --2- 172.21.15.181:0/3112109620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e42d00 0x559a19e430e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3b98015000 tx=0x7f3b9801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5123f634db64278 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:39.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.355+0000 7f3b9cd70700 1 -- 172.21.15.181:0/3112109620 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9802c070 con 0x559a19e42d00 2024-05-03T20:36:39.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.355+0000 7f3b9cd70700 1 -- 172.21.15.181:0/3112109620 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b98022070 con 0x559a19e42d00 2024-05-03T20:36:39.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.355+0000 7f3b9cd70700 1 -- 172.21.15.181:0/3112109620 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9801cab0 con 0x559a19e42d00 2024-05-03T20:36:39.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.356+0000 7f3bb2cecb80 1 -- 172.21.15.181:0/3112109620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e42d00 msgr2=0x559a19e430e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:39.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.356+0000 7f3bb2cecb80 1 --2- 172.21.15.181:0/3112109620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e42d00 0x559a19e430e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3b98015000 tx=0x7f3b9801c000 comp rx=0 tx=0).stop 2024-05-03T20:36:39.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.356+0000 7f3bb2cecb80 1 -- 172.21.15.181:0/3112109620 shutdown_connections 2024-05-03T20:36:39.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.356+0000 7f3bb2cecb80 1 --2- 172.21.15.181:0/3112109620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a19e3d8e0 0x559a19e3dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.356+0000 7f3bb2cecb80 1 --2- 172.21.15.181:0/3112109620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e42d00 0x559a19e430e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.356+0000 7f3bb2cecb80 1 --2- 172.21.15.181:0/3112109620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a19e43620 0x559a19e3d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.356+0000 7f3bb2cecb80 1 -- 172.21.15.181:0/3112109620 >> 172.21.15.181:0/3112109620 conn(0x559a19d65b20 msgr2=0x559a19e339a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:39.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.356+0000 7f3bb2cecb80 1 -- 172.21.15.181:0/3112109620 shutdown_connections 2024-05-03T20:36:39.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.356+0000 7f3bb2cecb80 1 -- 172.21.15.181:0/3112109620 wait complete. 2024-05-03T20:36:39.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.356+0000 7f3bb2cecb80 1 -- start start 2024-05-03T20:36:39.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.356+0000 7f3bb2cecb80 1 Processor -- start 2024-05-03T20:36:39.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.357+0000 7f3bb2cecb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e3d8e0 0x559a19e492a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:39.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.357+0000 7f3bb2cecb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a19e42d00 0x559a19e46640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:39.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.357+0000 7f3bb2cecb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a19e43620 0x559a19e46b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:39.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.357+0000 7f3bb2cecb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559a19e49ff0 con 0x559a19e3d8e0 2024-05-03T20:36:39.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.357+0000 7f3bb2cecb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559a19d44e80 con 0x559a19e43620 2024-05-03T20:36:39.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.357+0000 7f3bb2cecb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559a19e4a170 con 0x559a19e42d00 2024-05-03T20:36:39.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.357+0000 7f3bb2cecb80 10 client.0 _ll_register_callbacks cb 0x559a19d1f560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T20:36:39.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.357+0000 7f3ba61f5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e3d8e0 0x559a19e492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:39.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.357+0000 7f3ba61f5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e3d8e0 0x559a19e492a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36464/0 (socket says 172.21.15.181:36464) 2024-05-03T20:36:39.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.357+0000 7f3ba61f5700 1 -- 172.21.15.181:0/3468512047 learned_addr learned my addr 172.21.15.181:0/3468512047 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:39.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.357+0000 7f3ba69f6700 1 --2- 172.21.15.181:0/3468512047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a19e43620 0x559a19e46b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:39.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.357+0000 7f3ba59f4700 1 --2- 172.21.15.181:0/3468512047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a19e42d00 0x559a19e46640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:39.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.357+0000 7f3ba61f5700 1 -- 172.21.15.181:0/3468512047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a19e42d00 msgr2=0x559a19e46640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:39.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.357+0000 7f3ba61f5700 1 --2- 172.21.15.181:0/3468512047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a19e42d00 0x559a19e46640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.357+0000 7f3ba61f5700 1 -- 172.21.15.181:0/3468512047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a19e43620 msgr2=0x559a19e46b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:39.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.357+0000 7f3ba61f5700 1 --2- 172.21.15.181:0/3468512047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a19e43620 0x559a19e46b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.357+0000 7f3ba61f5700 1 -- 172.21.15.181:0/3468512047 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a19d42710 con 0x559a19e3d8e0 2024-05-03T20:36:39.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.357+0000 7f3ba69f6700 1 --2- 172.21.15.181:0/3468512047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a19e43620 0x559a19e46b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:36:39.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.357+0000 7f3ba61f5700 1 --2- 172.21.15.181:0/3468512047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e3d8e0 0x559a19e492a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3b980077c0 tx=0x7f3b980061b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfad8418f2c47027 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:39.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.357+0000 7f3b85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-03T20:36:39.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.357+0000 7f3b85ffb700 1 -- 172.21.15.181:0/3468512047 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98038030 con 0x559a19e3d8e0 2024-05-03T20:36:39.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.357+0000 7f3b85ffb700 1 -- 172.21.15.181:0/3468512047 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f3b9801c770 con 0x559a19e3d8e0 2024-05-03T20:36:39.362 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.357+0000 7f3bb2cecb80 1 -- 172.21.15.181:0/3468512047 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b98022070 con 0x559a19e3d8e0 2024-05-03T20:36:39.362 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.358+0000 7f3b85ffb700 1 -- 172.21.15.181:0/3468512047 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3b9802c040 con 0x559a19e3d8e0 2024-05-03T20:36:39.362 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.358+0000 7f3bb2cecb80 10 client.15114 fetch_fsmap learned FSMap version 27 2024-05-03T20:36:39.362 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.358+0000 7f3bb2cecb80 1 -- 172.21.15.181:0/3468512047 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559a19e547d0 con 0x559a19e3d8e0 2024-05-03T20:36:39.362 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.358+0000 7f3b85ffb700 1 -- 172.21.15.181:0/3468512047 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b98022070 con 0x559a19e3d8e0 2024-05-03T20:36:39.362 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.358+0000 7f3b85ffb700 1 -- 172.21.15.181:0/3468512047 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3b98027070 con 0x559a19e3d8e0 2024-05-03T20:36:39.362 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.358+0000 7f3bb2cecb80 10 client.15114 fetch_fsmap finished waiting for FSMap version 27 2024-05-03T20:36:39.362 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.358+0000 7f3bb2cecb80 10 client.15114 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-03T20:36:39.362 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.358+0000 7f3bb2cecb80 20 client.15114 populate_metadata read hostname 'smithi181' 2024-05-03T20:36:39.362 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.358+0000 7f3bb2cecb80 1 --2- 172.21.15.181:0/3468512047 >> [v2:172.21.15.196:6836/1529420259,v1:172.21.15.196:6837/1529420259] conn(0x559a19e55b20 0x559a19e57cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:39.362 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.358+0000 7f3bb2cecb80 4 client.15114 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T20:36:39.362 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.358+0000 7f3bb2cecb80 1 -- 172.21.15.181:0/3468512047 --> [v2:172.21.15.196:6836/1529420259,v1:172.21.15.196:6837/1529420259] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a19d992d0 con 0x559a19e55b20 2024-05-03T20:36:39.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.358+0000 7f3ba59f4700 1 --2- 172.21.15.181:0/3468512047 >> [v2:172.21.15.196:6836/1529420259,v1:172.21.15.196:6837/1529420259] conn(0x559a19e55b20 0x559a19e57cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:39.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.359+0000 7f3b85ffb700 10 client.15114 ms_handle_connect on v2:172.21.15.196:6836/1529420259 2024-05-03T20:36:39.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.359+0000 7f3ba59f4700 1 --2- 172.21.15.181:0/3468512047 >> [v2:172.21.15.196:6836/1529420259,v1:172.21.15.196:6837/1529420259] conn(0x559a19e55b20 0x559a19e57cd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=82d2b67b244c024a server_cookie=c4c6c9bb1ab45b63 in_seq=0 out_seq=0 2024-05-03T20:36:39.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.359+0000 7f3b85ffb700 1 -- 172.21.15.181:0/3468512047 <== mds.2 v2:172.21.15.196:6836/1529420259 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559a19d992d0 con 0x559a19e55b20 2024-05-03T20:36:39.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.359+0000 7f3b85ffb700 10 client.15114 handle_command_reply: tid=0 2024-05-03T20:36:39.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.359+0000 7f3bb2cecb80 1 client.15114 shutdown 2024-05-03T20:36:39.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.360+0000 7f3bb2cecb80 1 -- 172.21.15.181:0/3468512047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e3d8e0 msgr2=0x559a19e492a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:39.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.360+0000 7f3bb2cecb80 1 --2- 172.21.15.181:0/3468512047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e3d8e0 0x559a19e492a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3b980077c0 tx=0x7f3b980061b0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.360+0000 7f3bb2cecb80 1 -- 172.21.15.181:0/3468512047 shutdown_connections 2024-05-03T20:36:39.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.360+0000 7f3bb2cecb80 1 --2- 172.21.15.181:0/3468512047 >> [v2:172.21.15.196:6836/1529420259,v1:172.21.15.196:6837/1529420259] conn(0x559a19e55b20 0x559a19e57cd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.360+0000 7f3bb2cecb80 1 --2- 172.21.15.181:0/3468512047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a19e43620 0x559a19e46b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.360+0000 7f3bb2cecb80 1 --2- 172.21.15.181:0/3468512047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e3d8e0 0x559a19e492a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.360+0000 7f3bb2cecb80 1 --2- 172.21.15.181:0/3468512047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a19e42d00 0x559a19e46640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.360+0000 7f3bb2cecb80 1 -- 172.21.15.181:0/3468512047 >> 172.21.15.181:0/3468512047 conn(0x559a19d65b20 msgr2=0x559a19e420d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:39.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.360+0000 7f3b85ffb700 0 client.15114 ms_handle_reset on v2:172.21.15.196:6836/1529420259 2024-05-03T20:36:39.364 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.360+0000 7f3bb2cecb80 1 -- 172.21.15.181:0/3468512047 shutdown_connections 2024-05-03T20:36:39.364 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.360+0000 7f3bb2cecb80 1 -- 172.21.15.181:0/3468512047 wait complete. 2024-05-03T20:36:39.364 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.360+0000 7f3bb2cecb80 20 client.15114 trim_cache size 0 max 16384 2024-05-03T20:36:39.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.380+0000 7f3bb2cecb80 1 Processor -- start 2024-05-03T20:36:39.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.380+0000 7f3bb2cecb80 1 -- start start 2024-05-03T20:36:39.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.381+0000 7f3bb2cecb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e3d8e0 0x559a19e3d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:39.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.381+0000 7f3bb2cecb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a19e42d00 0x559a19e3dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:39.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.381+0000 7f3bb2cecb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a19e43620 0x559a19e35420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:39.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.381+0000 7f3bb2cecb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559a19e56010 con 0x559a19e3d8e0 2024-05-03T20:36:39.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.381+0000 7f3bb2cecb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559a19e566e0 con 0x559a19e42d00 2024-05-03T20:36:39.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.381+0000 7f3bb2cecb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559a19e56db0 con 0x559a19e43620 2024-05-03T20:36:39.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.381+0000 7f3ba61f5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e3d8e0 0x559a19e3d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:39.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.381+0000 7f3ba61f5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e3d8e0 0x559a19e3d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36480/0 (socket says 172.21.15.181:36480) 2024-05-03T20:36:39.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.381+0000 7f3ba61f5700 1 -- 172.21.15.181:0/1548559223 learned_addr learned my addr 172.21.15.181:0/1548559223 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:39.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.381+0000 7f3ba59f4700 1 --2- 172.21.15.181:0/1548559223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a19e42d00 0x559a19e3dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:39.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.381+0000 7f3ba69f6700 1 --2- 172.21.15.181:0/1548559223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a19e43620 0x559a19e35420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:39.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.381+0000 7f3ba61f5700 1 -- 172.21.15.181:0/1548559223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a19e43620 msgr2=0x559a19e35420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:39.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.381+0000 7f3ba61f5700 1 --2- 172.21.15.181:0/1548559223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a19e43620 0x559a19e35420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.381+0000 7f3ba61f5700 1 -- 172.21.15.181:0/1548559223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a19e42d00 msgr2=0x559a19e3dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:39.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.381+0000 7f3ba61f5700 1 --2- 172.21.15.181:0/1548559223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a19e42d00 0x559a19e3dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.381+0000 7f3ba61f5700 1 -- 172.21.15.181:0/1548559223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a19e48ee0 con 0x559a19e3d8e0 2024-05-03T20:36:39.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.381+0000 7f3ba59f4700 1 --2- 172.21.15.181:0/1548559223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a19e42d00 0x559a19e3dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:36:39.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.381+0000 7f3ba61f5700 1 --2- 172.21.15.181:0/1548559223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e3d8e0 0x559a19e3d350 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3b98031c10 tx=0x7f3b9801c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53beb2c74a9d20ab server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:39.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.381+0000 7f3b87fff700 1 -- 172.21.15.181:0/1548559223 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98038070 con 0x559a19e3d8e0 2024-05-03T20:36:39.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.381+0000 7f3b87fff700 1 -- 172.21.15.181:0/1548559223 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b98022040 con 0x559a19e3d8e0 2024-05-03T20:36:39.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.381+0000 7f3b87fff700 1 -- 172.21.15.181:0/1548559223 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98007ae0 con 0x559a19e3d8e0 2024-05-03T20:36:39.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.382+0000 7f3bb2cecb80 1 -- 172.21.15.181:0/1548559223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e3d8e0 msgr2=0x559a19e3d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:39.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.382+0000 7f3bb2cecb80 1 --2- 172.21.15.181:0/1548559223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e3d8e0 0x559a19e3d350 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3b98031c10 tx=0x7f3b9801c580 comp rx=0 tx=0).stop 2024-05-03T20:36:39.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.382+0000 7f3bb2cecb80 1 -- 172.21.15.181:0/1548559223 shutdown_connections 2024-05-03T20:36:39.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.382+0000 7f3bb2cecb80 1 --2- 172.21.15.181:0/1548559223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a19e43620 0x559a19e35420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.382+0000 7f3bb2cecb80 1 --2- 172.21.15.181:0/1548559223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e3d8e0 0x559a19e3d350 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.382+0000 7f3bb2cecb80 1 --2- 172.21.15.181:0/1548559223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a19e42d00 0x559a19e3dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.382+0000 7f3bb2cecb80 1 -- 172.21.15.181:0/1548559223 >> 172.21.15.181:0/1548559223 conn(0x559a19d65b20 msgr2=0x559a19e32800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:39.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.382+0000 7f3bb2cecb80 1 -- 172.21.15.181:0/1548559223 shutdown_connections 2024-05-03T20:36:39.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.382+0000 7f3bb2cecb80 1 -- 172.21.15.181:0/1548559223 wait complete. 2024-05-03T20:36:39.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.382+0000 7f3bb2cecb80 1 -- start start 2024-05-03T20:36:39.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.383+0000 7f3bb2cecb80 1 Processor -- start 2024-05-03T20:36:39.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.383+0000 7f3bb2cecb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e3d8e0 0x559a19e61150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:39.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.383+0000 7f3bb2cecb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a19e42d00 0x559a19e61690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:39.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.383+0000 7f3bb2cecb80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a19e43620 0x559a19e5c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:39.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.383+0000 7f3bb2cecb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x559a19e57b50 con 0x559a19e3d8e0 2024-05-03T20:36:39.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.383+0000 7f3bb2cecb80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559a19e57480 con 0x559a19e43620 2024-05-03T20:36:39.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.383+0000 7f3bb2cecb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559a19e58220 con 0x559a19e42d00 2024-05-03T20:36:39.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.383+0000 7f3bb2cecb80 10 client.0 _ll_register_callbacks cb 0x559a19d1f560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T20:36:39.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.383+0000 7f3ba61f5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e3d8e0 0x559a19e61150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:39.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.383+0000 7f3ba61f5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e3d8e0 0x559a19e61150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36490/0 (socket says 172.21.15.181:36490) 2024-05-03T20:36:39.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.383+0000 7f3ba61f5700 1 -- 172.21.15.181:0/165253350 learned_addr learned my addr 172.21.15.181:0/165253350 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:39.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.383+0000 7f3ba59f4700 1 --2- 172.21.15.181:0/165253350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a19e42d00 0x559a19e61690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:39.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.383+0000 7f3ba69f6700 1 --2- 172.21.15.181:0/165253350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a19e43620 0x559a19e5c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:39.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.383+0000 7f3ba61f5700 1 -- 172.21.15.181:0/165253350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a19e42d00 msgr2=0x559a19e61690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:39.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.383+0000 7f3ba61f5700 1 --2- 172.21.15.181:0/165253350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a19e42d00 0x559a19e61690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.383+0000 7f3ba61f5700 1 -- 172.21.15.181:0/165253350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a19e43620 msgr2=0x559a19e5c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:39.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.383+0000 7f3ba61f5700 1 --2- 172.21.15.181:0/165253350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a19e43620 0x559a19e5c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.383+0000 7f3ba61f5700 1 -- 172.21.15.181:0/165253350 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a19e5c740 con 0x559a19e3d8e0 2024-05-03T20:36:39.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.383+0000 7f3ba69f6700 1 --2- 172.21.15.181:0/165253350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a19e43620 0x559a19e5c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:36:39.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.383+0000 7f3ba59f4700 1 --2- 172.21.15.181:0/165253350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a19e42d00 0x559a19e61690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:36:39.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.383+0000 7f3ba61f5700 1 --2- 172.21.15.181:0/165253350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e3d8e0 0x559a19e61150 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009a20 tx=0x7f3b98009cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4a779b5cbb862c server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:39.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.383+0000 7f3b85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-03T20:36:39.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.383+0000 7f3b85ffb700 1 -- 172.21.15.181:0/165253350 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98038070 con 0x559a19e3d8e0 2024-05-03T20:36:39.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.384+0000 7f3b85ffb700 1 -- 172.21.15.181:0/165253350 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f3b980077c0 con 0x559a19e3d8e0 2024-05-03T20:36:39.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.384+0000 7f3bb2cecb80 1 -- 172.21.15.181:0/165253350 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b98022040 con 0x559a19e3d8e0 2024-05-03T20:36:39.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.384+0000 7f3b85ffb700 1 -- 172.21.15.181:0/165253350 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3b980317e0 con 0x559a19e3d8e0 2024-05-03T20:36:39.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.384+0000 7f3bb2cecb80 10 client.15120 fetch_fsmap learned FSMap version 27 2024-05-03T20:36:39.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.384+0000 7f3bb2cecb80 1 -- 172.21.15.181:0/165253350 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559a19ec2610 con 0x559a19e3d8e0 2024-05-03T20:36:39.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.384+0000 7f3b85ffb700 1 -- 172.21.15.181:0/165253350 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b98022040 con 0x559a19e3d8e0 2024-05-03T20:36:39.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.384+0000 7f3b85ffb700 1 -- 172.21.15.181:0/165253350 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3b98027070 con 0x559a19e3d8e0 2024-05-03T20:36:39.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.384+0000 7f3bb2cecb80 10 client.15120 fetch_fsmap finished waiting for FSMap version 27 2024-05-03T20:36:39.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.384+0000 7f3bb2cecb80 10 client.15120 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-03T20:36:39.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.384+0000 7f3bb2cecb80 20 client.15120 populate_metadata read hostname 'smithi181' 2024-05-03T20:36:39.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.384+0000 7f3bb2cecb80 1 --2- 172.21.15.181:0/165253350 >> [v2:172.21.15.196:6836/1529420259,v1:172.21.15.196:6837/1529420259] conn(0x559a19ec6de0 0x559a19ec5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:39.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.384+0000 7f3bb2cecb80 4 client.15120 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T20:36:39.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.384+0000 7f3bb2cecb80 1 -- 172.21.15.181:0/165253350 --> [v2:172.21.15.196:6836/1529420259,v1:172.21.15.196:6837/1529420259] -- command(tid 0: {"prefix": "status"}) -- 0x559a19d93370 con 0x559a19ec6de0 2024-05-03T20:36:39.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.384+0000 7f3ba59f4700 1 --2- 172.21.15.181:0/165253350 >> [v2:172.21.15.196:6836/1529420259,v1:172.21.15.196:6837/1529420259] conn(0x559a19ec6de0 0x559a19ec5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:39.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.385+0000 7f3ba59f4700 1 --2- 172.21.15.181:0/165253350 >> [v2:172.21.15.196:6836/1529420259,v1:172.21.15.196:6837/1529420259] conn(0x559a19ec6de0 0x559a19ec5af0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fe80f1fc0d1dffaf server_cookie=28fd78fd0e2f1d7a in_seq=0 out_seq=0 2024-05-03T20:36:39.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.385+0000 7f3b85ffb700 10 client.15120 ms_handle_connect on v2:172.21.15.196:6836/1529420259 2024-05-03T20:36:39.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.385+0000 7f3b85ffb700 1 -- 172.21.15.181:0/165253350 <== mds.2 v2:172.21.15.196:6836/1529420259 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559a19d93370 con 0x559a19ec6de0 2024-05-03T20:36:39.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.385+0000 7f3b85ffb700 10 client.15120 handle_command_reply: tid=0 2024-05-03T20:36:39.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.385+0000 7f3bb2cecb80 1 client.15120 shutdown 2024-05-03T20:36:39.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.385+0000 7f3bb2cecb80 1 -- 172.21.15.181:0/165253350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e3d8e0 msgr2=0x559a19e61150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:39.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.385+0000 7f3bb2cecb80 1 --2- 172.21.15.181:0/165253350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e3d8e0 0x559a19e61150 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009a20 tx=0x7f3b98009cc0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.385+0000 7f3bb2cecb80 1 -- 172.21.15.181:0/165253350 shutdown_connections 2024-05-03T20:36:39.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.385+0000 7f3bb2cecb80 1 --2- 172.21.15.181:0/165253350 >> [v2:172.21.15.196:6836/1529420259,v1:172.21.15.196:6837/1529420259] conn(0x559a19ec6de0 0x559a19ec5af0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.385+0000 7f3bb2cecb80 1 --2- 172.21.15.181:0/165253350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a19e43620 0x559a19e5c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.385+0000 7f3bb2cecb80 1 --2- 172.21.15.181:0/165253350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559a19e3d8e0 0x559a19e61150 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.385+0000 7f3bb2cecb80 1 --2- 172.21.15.181:0/165253350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a19e42d00 0x559a19e61690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.385+0000 7f3bb2cecb80 1 -- 172.21.15.181:0/165253350 >> 172.21.15.181:0/165253350 conn(0x559a19d65b20 msgr2=0x559a19e43a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:39.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.385+0000 7f3b85ffb700 0 client.15120 ms_handle_reset on v2:172.21.15.196:6836/1529420259 2024-05-03T20:36:39.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.385+0000 7f3bb2cecb80 1 -- 172.21.15.181:0/165253350 shutdown_connections 2024-05-03T20:36:39.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.385+0000 7f3bb2cecb80 1 -- 172.21.15.181:0/165253350 wait complete. 2024-05-03T20:36:39.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.385+0000 7f3bb2cecb80 20 client.15120 trim_cache size 0 max 16384 2024-05-03T20:36:39.390 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-05-03T20:36:39.390 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "1cd058ee-098b-11ef-bc97-c7b262605968", 2024-05-03T20:36:39.390 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 2, 2024-05-03T20:36:39.391 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 24464, 2024-05-03T20:36:39.391 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2024-05-03T20:36:39.391 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2024-05-03T20:36:39.391 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2024-05-03T20:36:39.391 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 163.927180198, 2024-05-03T20:36:39.391 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 27, 2024-05-03T20:36:39.391 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 83, 2024-05-03T20:36:39.391 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 81, 2024-05-03T20:36:39.391 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 203.72788858499999 2024-05-03T20:36:39.391 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-05-03T20:36:39.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.386+0000 7f3ba8459700 1 -- 172.21.15.181:0/3758626319 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3b880b2910 msgr2=0x7f3b880b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:39.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.386+0000 7f3ba8459700 1 --2- 172.21.15.181:0/3758626319 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3b880b2910 0x7f3b880b4dd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800d010 tx=0x7f3b9800b040 comp rx=0 tx=0).stop 2024-05-03T20:36:39.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.386+0000 7f3ba8459700 1 -- 172.21.15.181:0/3758626319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ba0104100 msgr2=0x7f3ba019fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:39.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.386+0000 7f3ba8459700 1 --2- 172.21.15.181:0/3758626319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ba0104100 0x7f3ba019fd40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3b9400c920 tx=0x7f3b9400cce0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.386+0000 7f3ba8459700 1 -- 172.21.15.181:0/3758626319 shutdown_connections 2024-05-03T20:36:39.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.386+0000 7f3ba8459700 1 --2- 172.21.15.181:0/3758626319 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f3b880b2910 0x7f3b880b4dd0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.386+0000 7f3ba8459700 1 --2- 172.21.15.181:0/3758626319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ba0108850 0x7f3ba01999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.386+0000 7f3ba8459700 1 --2- 172.21.15.181:0/3758626319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ba0103750 0x7f3ba019f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.386+0000 7f3ba8459700 1 --2- 172.21.15.181:0/3758626319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ba0104100 0x7f3ba019fd40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:39.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.386+0000 7f3ba8459700 1 -- 172.21.15.181:0/3758626319 >> 172.21.15.181:0/3758626319 conn(0x7f3ba007a600 msgr2=0x7f3ba0101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:39.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.386+0000 7f3ba8459700 1 -- 172.21.15.181:0/3758626319 shutdown_connections 2024-05-03T20:36:39.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:39.386+0000 7f3ba8459700 1 -- 172.21.15.181:0/3758626319 wait complete. 2024-05-03T20:36:39.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:39 smithi181 ceph-mon[29801]: pgmap v375: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:36:39.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:39 smithi196 ceph-mon[32858]: pgmap v375: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:36:39.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:39 smithi204 ceph-mon[33595]: pgmap v375: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:36:40.251 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-03T20:36:40.492 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:36:41.646 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:41 smithi181 ceph-mon[29801]: pgmap v376: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:36:41.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:41 smithi196 ceph-mon[32858]: pgmap v376: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:36:41.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:41 smithi204 ceph-mon[33595]: pgmap v376: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:36:42.161 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.160+0000 7f0cbcaee700 1 -- 172.21.15.181:0/1762302593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cb8108410 msgr2=0x7f0cb81087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:42.161 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.160+0000 7f0cbcaee700 1 --2- 172.21.15.181:0/1762302593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cb8108410 0x7f0cb81087f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8009990 tx=0x7f0ca8009ca0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.161 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.160+0000 7f0cbcaee700 1 -- 172.21.15.181:0/1762302593 shutdown_connections 2024-05-03T20:36:42.161 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.160+0000 7f0cbcaee700 1 --2- 172.21.15.181:0/1762302593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cb8102d60 0x7f0cb810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.161 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.160+0000 7f0cbcaee700 1 --2- 172.21.15.181:0/1762302593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cb8108410 0x7f0cb81087f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.161 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.160+0000 7f0cbcaee700 1 --2- 172.21.15.181:0/1762302593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cb81023a0 0x7f0cb8102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.161 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.160+0000 7f0cbcaee700 1 -- 172.21.15.181:0/1762302593 >> 172.21.15.181:0/1762302593 conn(0x7f0cb8079350 msgr2=0x7f0cb8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:42.162 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.160+0000 7f0cbcaee700 1 -- 172.21.15.181:0/1762302593 shutdown_connections 2024-05-03T20:36:42.162 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.160+0000 7f0cbcaee700 1 -- 172.21.15.181:0/1762302593 wait complete. 2024-05-03T20:36:42.162 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.161+0000 7f0cbcaee700 1 Processor -- start 2024-05-03T20:36:42.162 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.161+0000 7f0cbcaee700 1 -- start start 2024-05-03T20:36:42.162 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.161+0000 7f0cbcaee700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cb81023a0 0x7f0cb8198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:42.162 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.161+0000 7f0cbcaee700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cb8102d60 0x7f0cb8198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:42.162 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.161+0000 7f0cbcaee700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cb8108410 0x7f0cb81992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:42.162 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.161+0000 7f0cbcaee700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb81125b0 con 0x7f0cb8102d60 2024-05-03T20:36:42.162 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.161+0000 7f0cbcaee700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb8112430 con 0x7f0cb81023a0 2024-05-03T20:36:42.163 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.161+0000 7f0cbcaee700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb8112730 con 0x7f0cb8108410 2024-05-03T20:36:42.163 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.161+0000 7f0cb5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cb8102d60 0x7f0cb8198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:42.163 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.161+0000 7f0cb5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cb8102d60 0x7f0cb8198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36514/0 (socket says 172.21.15.181:36514) 2024-05-03T20:36:42.163 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.161+0000 7f0cb5d9b700 1 -- 172.21.15.181:0/2266187046 learned_addr learned my addr 172.21.15.181:0/2266187046 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:42.163 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.162+0000 7f0cb659c700 1 --2- 172.21.15.181:0/2266187046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cb81023a0 0x7f0cb8198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:42.163 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.162+0000 7f0cb5d9b700 1 -- 172.21.15.181:0/2266187046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cb8108410 msgr2=0x7f0cb81992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:42.163 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.162+0000 7f0cb5d9b700 1 --2- 172.21.15.181:0/2266187046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cb8108410 0x7f0cb81992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.163 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.162+0000 7f0cb5d9b700 1 -- 172.21.15.181:0/2266187046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cb81023a0 msgr2=0x7f0cb8198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:42.163 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.162+0000 7f0cb5d9b700 1 --2- 172.21.15.181:0/2266187046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cb81023a0 0x7f0cb8198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.163 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.162+0000 7f0cb5d9b700 1 -- 172.21.15.181:0/2266187046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb81a8070 con 0x7f0cb8102d60 2024-05-03T20:36:42.164 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.162+0000 7f0cb5d9b700 1 --2- 172.21.15.181:0/2266187046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cb8102d60 0x7f0cb8198da0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00d950 tx=0x7f0cac00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:42.164 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.162+0000 7f0ca77fe700 1 -- 172.21.15.181:0/2266187046 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac016070 con 0x7f0cb8102d60 2024-05-03T20:36:42.164 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.162+0000 7f0ca77fe700 1 -- 172.21.15.181:0/2266187046 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cac009df0 con 0x7f0cb8102d60 2024-05-03T20:36:42.164 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.162+0000 7f0cbcaee700 1 -- 172.21.15.181:0/2266187046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb81a8360 con 0x7f0cb8102d60 2024-05-03T20:36:42.164 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.162+0000 7f0ca77fe700 1 -- 172.21.15.181:0/2266187046 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac00f620 con 0x7f0cb8102d60 2024-05-03T20:36:42.164 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.162+0000 7f0cbcaee700 1 -- 172.21.15.181:0/2266187046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb81a88a0 con 0x7f0cb8102d60 2024-05-03T20:36:42.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.166+0000 7f0ca77fe700 1 -- 172.21.15.181:0/2266187046 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f0cac019070 con 0x7f0cb8102d60 2024-05-03T20:36:42.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.166+0000 7f0cc7381b80 1 Processor -- start 2024-05-03T20:36:42.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.167+0000 7f0cc7381b80 1 -- start start 2024-05-03T20:36:42.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.167+0000 7f0ca77fe700 1 --2- 172.21.15.181:0/2266187046 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f0ca00b2910 0x7f0ca00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:42.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.167+0000 7f0cc7381b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff011fdc10 0x55ff011fdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:42.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.167+0000 7f0cc7381b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff011fe530 0x55ff01201cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:42.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.167+0000 7f0cc7381b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff01202230 0x55ff01202690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:42.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.167+0000 7f0ca77fe700 1 -- 172.21.15.181:0/2266187046 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f0cac0e1c20 con 0x7f0cb8102d60 2024-05-03T20:36:42.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.167+0000 7f0cc7381b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ff01150810 con 0x55ff01202230 2024-05-03T20:36:42.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.167+0000 7f0cb6d9d700 1 --2- 172.21.15.181:0/2266187046 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f0ca00b2910 0x7f0ca00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:42.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.167+0000 7f0cb6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff01202230 0x55ff01202690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:42.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.167+0000 7f0cb6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff01202230 0x55ff01202690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36524/0 (socket says 172.21.15.181:36524) 2024-05-03T20:36:42.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.167+0000 7f0cb6d9d700 1 -- 172.21.15.181:0/302567388 learned_addr learned my addr 172.21.15.181:0/302567388 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:42.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.167+0000 7f0cc7381b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ff010ef770 con 0x55ff011fdc10 2024-05-03T20:36:42.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.167+0000 7f0cc7381b80 1 -- 172.21.15.181:0/302567388 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ff0114c7d0 con 0x55ff011fe530 2024-05-03T20:36:42.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.167+0000 7f0cb659c700 1 --2- 172.21.15.181:0/302567388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff011fdc10 0x55ff011fdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:42.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.167+0000 7f0cb5d9b700 1 --2- 172.21.15.181:0/302567388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff011fe530 0x55ff01201cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:42.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.167+0000 7f0cb659c700 1 -- 172.21.15.181:0/302567388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff011fe530 msgr2=0x55ff01201cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:42.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.167+0000 7f0cb659c700 1 --2- 172.21.15.181:0/302567388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff011fe530 0x55ff01201cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.167+0000 7f0cb659c700 1 -- 172.21.15.181:0/302567388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff01202230 msgr2=0x55ff01202690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:42.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.167+0000 7f0cb659c700 1 --2- 172.21.15.181:0/302567388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff01202230 0x55ff01202690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.167+0000 7f0cb659c700 1 -- 172.21.15.181:0/302567388 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff0109d880 con 0x55ff011fdc10 2024-05-03T20:36:42.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.167+0000 7f0cb6d9d700 1 --2- 172.21.15.181:0/302567388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff01202230 0x55ff01202690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T20:36:42.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.168+0000 7f0cb6d9d700 1 --2- 172.21.15.181:0/2266187046 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f0ca00b2910 0x7f0ca00b4dd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0cb00138a0 tx=0x7f0cb00091c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:42.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.168+0000 7f0cb659c700 1 --2- 172.21.15.181:0/302567388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff011fdc10 0x55ff011fdff0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0ca80059d0 tx=0x7f0ca8005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37335b053c6cdc03 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:42.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.168+0000 7f0ca4ff9700 1 -- 172.21.15.181:0/302567388 <== mon.2 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8003a60 con 0x55ff011fdc10 2024-05-03T20:36:42.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.168+0000 7f0ca4ff9700 1 -- 172.21.15.181:0/302567388 <== mon.2 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ca8026070 con 0x55ff011fdc10 2024-05-03T20:36:42.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.168+0000 7f0ca4ff9700 1 -- 172.21.15.181:0/302567388 <== mon.2 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca800ba30 con 0x55ff011fdc10 2024-05-03T20:36:42.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.169+0000 7f0cc7381b80 1 -- 172.21.15.181:0/302567388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff011fdc10 msgr2=0x55ff011fdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:42.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.169+0000 7f0cc7381b80 1 --2- 172.21.15.181:0/302567388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff011fdc10 0x55ff011fdff0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0ca80059d0 tx=0x7f0ca8005c70 comp rx=0 tx=0).stop 2024-05-03T20:36:42.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.169+0000 7f0cc7381b80 1 -- 172.21.15.181:0/302567388 shutdown_connections 2024-05-03T20:36:42.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.169+0000 7f0cc7381b80 1 --2- 172.21.15.181:0/302567388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff01202230 0x55ff01202690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.169+0000 7f0cc7381b80 1 --2- 172.21.15.181:0/302567388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff011fdc10 0x55ff011fdff0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.169+0000 7f0cc7381b80 1 --2- 172.21.15.181:0/302567388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff011fe530 0x55ff01201cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.169+0000 7f0cc7381b80 1 -- 172.21.15.181:0/302567388 >> 172.21.15.181:0/302567388 conn(0x55ff0112a0b0 msgr2=0x55ff011f8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:42.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.169+0000 7f0cc7381b80 1 -- 172.21.15.181:0/302567388 shutdown_connections 2024-05-03T20:36:42.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.169+0000 7f0cc7381b80 1 -- 172.21.15.181:0/302567388 wait complete. 2024-05-03T20:36:42.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.169+0000 7f0cc7381b80 1 -- start start 2024-05-03T20:36:42.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.169+0000 7f0cc7381b80 1 Processor -- start 2024-05-03T20:36:42.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.170+0000 7f0cc7381b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff011fdc10 0x55ff01209670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:42.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.170+0000 7f0cc7381b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff011fe530 0x55ff01206a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:42.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.170+0000 7f0cc7381b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff01202230 0x55ff01206f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:42.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.170+0000 7f0cc7381b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ff0120a430 con 0x55ff011fe530 2024-05-03T20:36:42.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.170+0000 7f0cc7381b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ff010b6be0 con 0x55ff011fdc10 2024-05-03T20:36:42.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.170+0000 7f0cc7381b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ff0120a5b0 con 0x55ff01202230 2024-05-03T20:36:42.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.170+0000 7f0cc7381b80 10 client.0 _ll_register_callbacks cb 0x55ff010e88d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T20:36:42.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.170+0000 7f0cb5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff011fe530 0x55ff01206a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:42.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.170+0000 7f0cb5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff011fe530 0x55ff01206a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36534/0 (socket says 172.21.15.181:36534) 2024-05-03T20:36:42.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.170+0000 7f0cb5d9b700 1 -- 172.21.15.181:0/347500864 learned_addr learned my addr 172.21.15.181:0/347500864 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:42.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.170+0000 7f0cb6d9d700 1 --2- 172.21.15.181:0/347500864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff01202230 0x55ff01206f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:42.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.170+0000 7f0cb659c700 1 --2- 172.21.15.181:0/347500864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff011fdc10 0x55ff01209670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:42.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.170+0000 7f0cb5d9b700 1 -- 172.21.15.181:0/347500864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff01202230 msgr2=0x55ff01206f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:42.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.170+0000 7f0cb5d9b700 1 --2- 172.21.15.181:0/347500864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff01202230 0x55ff01206f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.170+0000 7f0cb5d9b700 1 -- 172.21.15.181:0/347500864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff011fdc10 msgr2=0x55ff01209670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:42.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.170+0000 7f0cb5d9b700 1 --2- 172.21.15.181:0/347500864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff011fdc10 0x55ff01209670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.170+0000 7f0cb5d9b700 1 -- 172.21.15.181:0/347500864 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff010ec640 con 0x55ff011fe530 2024-05-03T20:36:42.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.170+0000 7f0cb5d9b700 1 --2- 172.21.15.181:0/347500864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff011fe530 0x55ff01206a10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0a50b0 tx=0x7f0cac0a8c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b8fed585e457fb server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:42.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.170+0000 7f0c9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-03T20:36:42.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.170+0000 7f0c9dffb700 1 -- 172.21.15.181:0/347500864 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0a9080 con 0x55ff011fe530 2024-05-03T20:36:42.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.171+0000 7f0c9dffb700 1 -- 172.21.15.181:0/347500864 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f0cac00ea40 con 0x55ff011fe530 2024-05-03T20:36:42.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.171+0000 7f0cc7381b80 1 -- 172.21.15.181:0/347500864 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ca8026070 con 0x55ff011fe530 2024-05-03T20:36:42.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.171+0000 7f0c9dffb700 1 -- 172.21.15.181:0/347500864 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0cac00ec40 con 0x55ff011fe530 2024-05-03T20:36:42.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.171+0000 7f0cc7381b80 10 client.15132 fetch_fsmap learned FSMap version 27 2024-05-03T20:36:42.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.171+0000 7f0cc7381b80 1 -- 172.21.15.181:0/347500864 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ff01214c30 con 0x55ff011fe530 2024-05-03T20:36:42.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.171+0000 7f0c9dffb700 1 -- 172.21.15.181:0/347500864 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ca8026070 con 0x55ff011fe530 2024-05-03T20:36:42.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.171+0000 7f0c9dffb700 1 -- 172.21.15.181:0/347500864 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0cac0b1a50 con 0x55ff011fe530 2024-05-03T20:36:42.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.171+0000 7f0cc7381b80 10 client.15132 fetch_fsmap finished waiting for FSMap version 27 2024-05-03T20:36:42.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.171+0000 7f0cc7381b80 10 client.15132 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-03T20:36:42.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.171+0000 7f0cc7381b80 20 client.15132 populate_metadata read hostname 'smithi181' 2024-05-03T20:36:42.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.171+0000 7f0cc7381b80 1 --2- 172.21.15.181:0/347500864 >> [v2:172.21.15.204:6838/1644828559,v1:172.21.15.204:6839/1644828559] conn(0x55ff01217210 0x55ff01218680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:42.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.171+0000 7f0cc7381b80 4 client.15132 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T20:36:42.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.171+0000 7f0cc7381b80 1 -- 172.21.15.181:0/347500864 --> [v2:172.21.15.204:6838/1644828559,v1:172.21.15.204:6839/1644828559] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff0115e1f0 con 0x55ff01217210 2024-05-03T20:36:42.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.172+0000 7f0cb659c700 1 --2- 172.21.15.181:0/347500864 >> [v2:172.21.15.204:6838/1644828559,v1:172.21.15.204:6839/1644828559] conn(0x55ff01217210 0x55ff01218680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:42.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.172+0000 7f0cb659c700 1 --2- 172.21.15.181:0/347500864 >> [v2:172.21.15.204:6838/1644828559,v1:172.21.15.204:6839/1644828559] conn(0x55ff01217210 0x55ff01218680 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b70bc0b8ed07296d server_cookie=42f80aea494a30ee in_seq=0 out_seq=0 2024-05-03T20:36:42.178 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.172+0000 7f0c9dffb700 10 client.15132 ms_handle_connect on v2:172.21.15.204:6838/1644828559 2024-05-03T20:36:42.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.173+0000 7f0c9dffb700 1 -- 172.21.15.181:0/347500864 <== mds.1 v2:172.21.15.204:6838/1644828559 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ff0115e1f0 con 0x55ff01217210 2024-05-03T20:36:42.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.173+0000 7f0c9dffb700 10 client.15132 handle_command_reply: tid=0 2024-05-03T20:36:42.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.173+0000 7f0cc7381b80 1 client.15132 shutdown 2024-05-03T20:36:42.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.173+0000 7f0cc7381b80 1 -- 172.21.15.181:0/347500864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff011fe530 msgr2=0x55ff01206a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:42.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.173+0000 7f0cc7381b80 1 --2- 172.21.15.181:0/347500864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff011fe530 0x55ff01206a10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0a50b0 tx=0x7f0cac0a8c40 comp rx=0 tx=0).stop 2024-05-03T20:36:42.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.173+0000 7f0cc7381b80 1 -- 172.21.15.181:0/347500864 shutdown_connections 2024-05-03T20:36:42.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.173+0000 7f0cc7381b80 1 --2- 172.21.15.181:0/347500864 >> [v2:172.21.15.204:6838/1644828559,v1:172.21.15.204:6839/1644828559] conn(0x55ff01217210 0x55ff01218680 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.173+0000 7f0cc7381b80 1 --2- 172.21.15.181:0/347500864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff01202230 0x55ff01206f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.173+0000 7f0cc7381b80 1 --2- 172.21.15.181:0/347500864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff011fdc10 0x55ff01209670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.173+0000 7f0c9dffb700 0 client.15132 ms_handle_reset on v2:172.21.15.204:6838/1644828559 2024-05-03T20:36:42.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.173+0000 7f0cc7381b80 1 --2- 172.21.15.181:0/347500864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff011fe530 0x55ff01206a10 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.173+0000 7f0cc7381b80 1 -- 172.21.15.181:0/347500864 >> 172.21.15.181:0/347500864 conn(0x55ff0112a0b0 msgr2=0x55ff012036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:42.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.173+0000 7f0cc7381b80 1 -- 172.21.15.181:0/347500864 shutdown_connections 2024-05-03T20:36:42.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.173+0000 7f0cc7381b80 1 -- 172.21.15.181:0/347500864 wait complete. 2024-05-03T20:36:42.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.173+0000 7f0cc7381b80 20 client.15132 trim_cache size 0 max 16384 2024-05-03T20:36:42.198 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.194+0000 7f0cc7381b80 1 Processor -- start 2024-05-03T20:36:42.198 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.194+0000 7f0cc7381b80 1 -- start start 2024-05-03T20:36:42.198 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.194+0000 7f0cc7381b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff011fdc10 0x55ff01203620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:42.198 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.194+0000 7f0cc7381b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff011fe530 0x55ff011fd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:42.198 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.194+0000 7f0cc7381b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff01202230 0x55ff01209020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:42.198 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.194+0000 7f0cc7381b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ff012169c0 con 0x55ff011fe530 2024-05-03T20:36:42.198 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.194+0000 7f0cc7381b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ff01217090 con 0x55ff011fdc10 2024-05-03T20:36:42.199 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.194+0000 7f0cc7381b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ff01217760 con 0x55ff01202230 2024-05-03T20:36:42.199 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.194+0000 7f0cb5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff011fe530 0x55ff011fd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:42.199 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.194+0000 7f0cb5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff011fe530 0x55ff011fd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36540/0 (socket says 172.21.15.181:36540) 2024-05-03T20:36:42.199 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.194+0000 7f0cb5d9b700 1 -- 172.21.15.181:0/3805642733 learned_addr learned my addr 172.21.15.181:0/3805642733 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:42.199 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.194+0000 7f0cb659c700 1 --2- 172.21.15.181:0/3805642733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff011fdc10 0x55ff01203620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:42.199 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.194+0000 7f0cb6d9d700 1 --2- 172.21.15.181:0/3805642733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff01202230 0x55ff01209020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:42.199 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.194+0000 7f0cb5d9b700 1 -- 172.21.15.181:0/3805642733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff01202230 msgr2=0x55ff01209020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:42.199 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.194+0000 7f0cb5d9b700 1 --2- 172.21.15.181:0/3805642733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff01202230 0x55ff01209020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.199 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.194+0000 7f0cb5d9b700 1 -- 172.21.15.181:0/3805642733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff011fdc10 msgr2=0x55ff01203620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:42.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.194+0000 7f0cb5d9b700 1 --2- 172.21.15.181:0/3805642733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff011fdc10 0x55ff01203620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.194+0000 7f0cb5d9b700 1 -- 172.21.15.181:0/3805642733 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff01213850 con 0x55ff011fe530 2024-05-03T20:36:42.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.194+0000 7f0cb5d9b700 1 --2- 172.21.15.181:0/3805642733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff011fe530 0x55ff011fd130 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0cac09f040 tx=0x7f0cac020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe830d50a1cc731 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:42.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.195+0000 7f0c9ffff700 1 -- 172.21.15.181:0/3805642733 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac06d070 con 0x55ff011fe530 2024-05-03T20:36:42.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.195+0000 7f0c9ffff700 1 -- 172.21.15.181:0/3805642733 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cac0a3360 con 0x55ff011fe530 2024-05-03T20:36:42.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.195+0000 7f0c9ffff700 1 -- 172.21.15.181:0/3805642733 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac010e10 con 0x55ff011fe530 2024-05-03T20:36:42.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.195+0000 7f0cc7381b80 1 -- 172.21.15.181:0/3805642733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff011fe530 msgr2=0x55ff011fd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:42.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.195+0000 7f0cc7381b80 1 --2- 172.21.15.181:0/3805642733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff011fe530 0x55ff011fd130 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0cac09f040 tx=0x7f0cac020000 comp rx=0 tx=0).stop 2024-05-03T20:36:42.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.195+0000 7f0cc7381b80 1 -- 172.21.15.181:0/3805642733 shutdown_connections 2024-05-03T20:36:42.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.195+0000 7f0cc7381b80 1 --2- 172.21.15.181:0/3805642733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff01202230 0x55ff01209020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.195+0000 7f0cc7381b80 1 --2- 172.21.15.181:0/3805642733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff011fdc10 0x55ff01203620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.195+0000 7f0cc7381b80 1 --2- 172.21.15.181:0/3805642733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff011fe530 0x55ff011fd130 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.195+0000 7f0cc7381b80 1 -- 172.21.15.181:0/3805642733 >> 172.21.15.181:0/3805642733 conn(0x55ff0112a0b0 msgr2=0x55ff011fca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:42.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.195+0000 7f0cc7381b80 1 -- 172.21.15.181:0/3805642733 shutdown_connections 2024-05-03T20:36:42.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.195+0000 7f0cc7381b80 1 -- 172.21.15.181:0/3805642733 wait complete. 2024-05-03T20:36:42.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.196+0000 7f0cc7381b80 1 -- start start 2024-05-03T20:36:42.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.196+0000 7f0cc7381b80 1 Processor -- start 2024-05-03T20:36:42.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.196+0000 7f0cc7381b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff011fdc10 0x55ff01220cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:42.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.196+0000 7f0cc7381b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff011fe530 0x55ff01221210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:42.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.196+0000 7f0cc7381b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff01202230 0x55ff01221790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:42.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.196+0000 7f0cc7381b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ff01218500 con 0x55ff01202230 2024-05-03T20:36:42.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.196+0000 7f0cc7381b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ff01217e30 con 0x55ff011fe530 2024-05-03T20:36:42.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.196+0000 7f0cc7381b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ff01218bd0 con 0x55ff011fdc10 2024-05-03T20:36:42.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.196+0000 7f0cc7381b80 10 client.0 _ll_register_callbacks cb 0x55ff010e88d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T20:36:42.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.196+0000 7f0cb6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff01202230 0x55ff01221790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:42.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.196+0000 7f0cb6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff01202230 0x55ff01221790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36554/0 (socket says 172.21.15.181:36554) 2024-05-03T20:36:42.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.196+0000 7f0cb6d9d700 1 -- 172.21.15.181:0/249093651 learned_addr learned my addr 172.21.15.181:0/249093651 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:42.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.196+0000 7f0cb659c700 1 --2- 172.21.15.181:0/249093651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff011fdc10 0x55ff01220cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:42.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.196+0000 7f0cb5d9b700 1 --2- 172.21.15.181:0/249093651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff011fe530 0x55ff01221210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:42.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.196+0000 7f0cb6d9d700 1 -- 172.21.15.181:0/249093651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff011fdc10 msgr2=0x55ff01220cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:42.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.196+0000 7f0cb6d9d700 1 --2- 172.21.15.181:0/249093651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff011fdc10 0x55ff01220cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.203 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.196+0000 7f0cb6d9d700 1 -- 172.21.15.181:0/249093651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff011fe530 msgr2=0x55ff01221210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:42.203 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.196+0000 7f0cb6d9d700 1 --2- 172.21.15.181:0/249093651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff011fe530 0x55ff01221210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.203 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.196+0000 7f0cb6d9d700 1 -- 172.21.15.181:0/249093651 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff01221fd0 con 0x55ff01202230 2024-05-03T20:36:42.203 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.196+0000 7f0cb659c700 1 --2- 172.21.15.181:0/249093651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff011fdc10 0x55ff01220cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T20:36:42.203 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.197+0000 7f0cb6d9d700 1 --2- 172.21.15.181:0/249093651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff01202230 0x55ff01221790 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0cb00042f0 tx=0x7f0cb0006d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbc11da2b827563 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:42.203 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.197+0000 7f0c9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-03T20:36:42.203 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.197+0000 7f0c9dffb700 1 -- 172.21.15.181:0/249093651 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0028070 con 0x55ff01202230 2024-05-03T20:36:42.203 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.197+0000 7f0c9dffb700 1 -- 172.21.15.181:0/249093651 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f0cb0024630 con 0x55ff01202230 2024-05-03T20:36:42.203 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.197+0000 7f0cc7381b80 1 -- 172.21.15.181:0/249093651 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cac0a3360 con 0x55ff01202230 2024-05-03T20:36:42.203 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.197+0000 7f0c9dffb700 1 -- 172.21.15.181:0/249093651 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0cb0034050 con 0x55ff01202230 2024-05-03T20:36:42.203 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.197+0000 7f0cc7381b80 10 client.15138 fetch_fsmap learned FSMap version 27 2024-05-03T20:36:42.203 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.197+0000 7f0cc7381b80 1 -- 172.21.15.181:0/249093651 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ff011ff810 con 0x55ff01202230 2024-05-03T20:36:42.203 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.197+0000 7f0c9dffb700 1 -- 172.21.15.181:0/249093651 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cac0a3360 con 0x55ff01202230 2024-05-03T20:36:42.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.197+0000 7f0c9dffb700 1 -- 172.21.15.181:0/249093651 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0cb0015040 con 0x55ff01202230 2024-05-03T20:36:42.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.197+0000 7f0cc7381b80 10 client.15138 fetch_fsmap finished waiting for FSMap version 27 2024-05-03T20:36:42.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.197+0000 7f0cc7381b80 10 client.15138 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-03T20:36:42.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.197+0000 7f0cc7381b80 20 client.15138 populate_metadata read hostname 'smithi181' 2024-05-03T20:36:42.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.197+0000 7f0cc7381b80 1 --2- 172.21.15.181:0/249093651 >> [v2:172.21.15.204:6838/1644828559,v1:172.21.15.204:6839/1644828559] conn(0x55ff012907d0 0x55ff0128f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:42.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.197+0000 7f0cc7381b80 4 client.15138 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T20:36:42.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.197+0000 7f0cc7381b80 1 -- 172.21.15.181:0/249093651 --> [v2:172.21.15.204:6838/1644828559,v1:172.21.15.204:6839/1644828559] -- command(tid 0: {"prefix": "status"}) -- 0x55ff01158290 con 0x55ff012907d0 2024-05-03T20:36:42.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.198+0000 7f0cb659c700 1 --2- 172.21.15.181:0/249093651 >> [v2:172.21.15.204:6838/1644828559,v1:172.21.15.204:6839/1644828559] conn(0x55ff012907d0 0x55ff0128f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:42.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.198+0000 7f0cb659c700 1 --2- 172.21.15.181:0/249093651 >> [v2:172.21.15.204:6838/1644828559,v1:172.21.15.204:6839/1644828559] conn(0x55ff012907d0 0x55ff0128f4e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe34cdd5809b971a server_cookie=ece5f344a5f8246f in_seq=0 out_seq=0 2024-05-03T20:36:42.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.198+0000 7f0c9dffb700 10 client.15138 ms_handle_connect on v2:172.21.15.204:6838/1644828559 2024-05-03T20:36:42.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.198+0000 7f0c9dffb700 1 -- 172.21.15.181:0/249093651 <== mds.1 v2:172.21.15.204:6838/1644828559 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ff01158290 con 0x55ff012907d0 2024-05-03T20:36:42.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.198+0000 7f0c9dffb700 10 client.15138 handle_command_reply: tid=0 2024-05-03T20:36:42.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.198+0000 7f0cc7381b80 1 client.15138 shutdown 2024-05-03T20:36:42.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.198+0000 7f0cc7381b80 1 -- 172.21.15.181:0/249093651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff01202230 msgr2=0x55ff01221790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:42.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.198+0000 7f0cc7381b80 1 --2- 172.21.15.181:0/249093651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff01202230 0x55ff01221790 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0cb00042f0 tx=0x7f0cb0006d00 comp rx=0 tx=0).stop 2024-05-03T20:36:42.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.198+0000 7f0cc7381b80 1 -- 172.21.15.181:0/249093651 shutdown_connections 2024-05-03T20:36:42.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.198+0000 7f0cc7381b80 1 --2- 172.21.15.181:0/249093651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff01202230 0x55ff01221790 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.198+0000 7f0cc7381b80 1 --2- 172.21.15.181:0/249093651 >> [v2:172.21.15.204:6838/1644828559,v1:172.21.15.204:6839/1644828559] conn(0x55ff012907d0 0x55ff0128f4e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.198+0000 7f0cc7381b80 1 --2- 172.21.15.181:0/249093651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff011fdc10 0x55ff01220cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.198+0000 7f0cc7381b80 1 --2- 172.21.15.181:0/249093651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ff011fe530 0x55ff01221210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.198+0000 7f0cc7381b80 1 -- 172.21.15.181:0/249093651 >> 172.21.15.181:0/249093651 conn(0x55ff0112a0b0 msgr2=0x55ff011fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:42.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.198+0000 7f0c9dffb700 0 client.15138 ms_handle_reset on v2:172.21.15.204:6838/1644828559 2024-05-03T20:36:42.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.198+0000 7f0cc7381b80 1 -- 172.21.15.181:0/249093651 shutdown_connections 2024-05-03T20:36:42.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.198+0000 7f0cc7381b80 1 -- 172.21.15.181:0/249093651 wait complete. 2024-05-03T20:36:42.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.199+0000 7f0cc7381b80 20 client.15138 trim_cache size 0 max 16384 2024-05-03T20:36:42.206 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-05-03T20:36:42.206 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "1cd058ee-098b-11ef-bc97-c7b262605968", 2024-05-03T20:36:42.206 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 1, 2024-05-03T20:36:42.206 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 24475, 2024-05-03T20:36:42.206 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2024-05-03T20:36:42.206 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2024-05-03T20:36:42.206 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2024-05-03T20:36:42.206 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 167.74647493399999, 2024-05-03T20:36:42.207 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 27, 2024-05-03T20:36:42.207 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 83, 2024-05-03T20:36:42.207 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 81, 2024-05-03T20:36:42.207 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 203.86011585200001 2024-05-03T20:36:42.207 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-05-03T20:36:42.208 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.199+0000 7f0cbcaee700 1 -- 172.21.15.181:0/2266187046 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f0ca00b2910 msgr2=0x7f0ca00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:42.208 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.199+0000 7f0cbcaee700 1 --2- 172.21.15.181:0/2266187046 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f0ca00b2910 0x7f0ca00b4dd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0cb00138a0 tx=0x7f0cb00091c0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.208 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.199+0000 7f0cbcaee700 1 -- 172.21.15.181:0/2266187046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cb8102d60 msgr2=0x7f0cb8198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:42.208 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.199+0000 7f0cbcaee700 1 --2- 172.21.15.181:0/2266187046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cb8102d60 0x7f0cb8198da0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00d950 tx=0x7f0cac00dc60 comp rx=0 tx=0).stop 2024-05-03T20:36:42.208 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.199+0000 7f0cbcaee700 1 -- 172.21.15.181:0/2266187046 shutdown_connections 2024-05-03T20:36:42.208 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.199+0000 7f0cbcaee700 1 --2- 172.21.15.181:0/2266187046 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f0ca00b2910 0x7f0ca00b4dd0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.208 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.199+0000 7f0cbcaee700 1 --2- 172.21.15.181:0/2266187046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cb8108410 0x7f0cb81992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.208 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.199+0000 7f0cbcaee700 1 --2- 172.21.15.181:0/2266187046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cb81023a0 0x7f0cb8198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.208 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.199+0000 7f0cbcaee700 1 --2- 172.21.15.181:0/2266187046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cb8102d60 0x7f0cb8198da0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:42.209 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.199+0000 7f0cbcaee700 1 -- 172.21.15.181:0/2266187046 >> 172.21.15.181:0/2266187046 conn(0x7f0cb8079350 msgr2=0x7f0cb8101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:42.209 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.199+0000 7f0cbcaee700 1 -- 172.21.15.181:0/2266187046 shutdown_connections 2024-05-03T20:36:42.209 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:42.199+0000 7f0cbcaee700 1 -- 172.21.15.181:0/2266187046 wait complete. 2024-05-03T20:36:42.856 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-03T20:36:43.098 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:36:43.499 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:43 smithi181 ceph-mon[29801]: pgmap v377: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-03T20:36:43.665 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:43 smithi204 ceph-mon[33595]: pgmap v377: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-03T20:36:43.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:43 smithi196 ceph-mon[32858]: pgmap v377: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-03T20:36:44.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.827+0000 7f239c832700 1 -- 172.21.15.181:0/3823171462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2398102390 msgr2=0x7f2398102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:44.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.827+0000 7f239c832700 1 --2- 172.21.15.181:0/3823171462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2398102390 0x7f2398102810 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f238c00b0a0 tx=0x7f238c00b3b0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.827+0000 7f239c832700 1 -- 172.21.15.181:0/3823171462 shutdown_connections 2024-05-03T20:36:44.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.827+0000 7f239c832700 1 --2- 172.21.15.181:0/3823171462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2398102d50 0x7f239810f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.827+0000 7f239c832700 1 --2- 172.21.15.181:0/3823171462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2398108400 0x7f23981087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.827+0000 7f239c832700 1 --2- 172.21.15.181:0/3823171462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2398102390 0x7f2398102810 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.827+0000 7f239c832700 1 -- 172.21.15.181:0/3823171462 >> 172.21.15.181:0/3823171462 conn(0x7f2398079310 msgr2=0x7f2398079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:44.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.827+0000 7f239c832700 1 -- 172.21.15.181:0/3823171462 shutdown_connections 2024-05-03T20:36:44.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.827+0000 7f239c832700 1 -- 172.21.15.181:0/3823171462 wait complete. 2024-05-03T20:36:44.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.828+0000 7f239c832700 1 Processor -- start 2024-05-03T20:36:44.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.828+0000 7f239c832700 1 -- start start 2024-05-03T20:36:44.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.828+0000 7f239c832700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2398102390 0x7f239810fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:44.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.828+0000 7f239c832700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2398102d50 0x7f2398110090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:44.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.828+0000 7f239c832700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2398108400 0x7f23981105d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:44.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.828+0000 7f239c832700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2398113720 con 0x7f2398108400 2024-05-03T20:36:44.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.828+0000 7f239c832700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f23981135a0 con 0x7f2398102d50 2024-05-03T20:36:44.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.828+0000 7f239c832700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f23981138a0 con 0x7f2398102390 2024-05-03T20:36:44.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.828+0000 7f2396d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2398108400 0x7f23981105d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:44.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.828+0000 7f2396d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2398108400 0x7f23981105d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36560/0 (socket says 172.21.15.181:36560) 2024-05-03T20:36:44.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.828+0000 7f2396d9d700 1 -- 172.21.15.181:0/2232755265 learned_addr learned my addr 172.21.15.181:0/2232755265 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:44.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.829+0000 7f2395d9b700 1 --2- 172.21.15.181:0/2232755265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2398102d50 0x7f2398110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:44.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.829+0000 7f239659c700 1 --2- 172.21.15.181:0/2232755265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2398102390 0x7f239810fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:44.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.829+0000 7f2396d9d700 1 -- 172.21.15.181:0/2232755265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2398102390 msgr2=0x7f239810fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:44.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.829+0000 7f2396d9d700 1 --2- 172.21.15.181:0/2232755265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2398102390 0x7f239810fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.829+0000 7f2396d9d700 1 -- 172.21.15.181:0/2232755265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2398102d50 msgr2=0x7f2398110090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:44.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.829+0000 7f2396d9d700 1 --2- 172.21.15.181:0/2232755265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2398102d50 0x7f2398110090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.829+0000 7f2396d9d700 1 -- 172.21.15.181:0/2232755265 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23981a8220 con 0x7f2398108400 2024-05-03T20:36:44.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.829+0000 7f2395d9b700 1 --2- 172.21.15.181:0/2232755265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2398102d50 0x7f2398110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:36:44.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.829+0000 7f2396d9d700 1 --2- 172.21.15.181:0/2232755265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2398108400 0x7f23981105d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f238800b550 tx=0x7f238800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:44.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.829+0000 7f23877fe700 1 -- 172.21.15.181:0/2232755265 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388015020 con 0x7f2398108400 2024-05-03T20:36:44.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.829+0000 7f23877fe700 1 -- 172.21.15.181:0/2232755265 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2388011ce0 con 0x7f2398108400 2024-05-03T20:36:44.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.829+0000 7f239c832700 1 -- 172.21.15.181:0/2232755265 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23981a8510 con 0x7f2398108400 2024-05-03T20:36:44.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.829+0000 7f23877fe700 1 -- 172.21.15.181:0/2232755265 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388010d40 con 0x7f2398108400 2024-05-03T20:36:44.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.829+0000 7f239c832700 1 -- 172.21.15.181:0/2232755265 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23981a8a50 con 0x7f2398108400 2024-05-03T20:36:44.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.831+0000 7f23877fe700 1 -- 172.21.15.181:0/2232755265 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f2388018020 con 0x7f2398108400 2024-05-03T20:36:44.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.832+0000 7f23877fe700 1 --2- 172.21.15.181:0/2232755265 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f23800b2990 0x7f23800b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:44.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.832+0000 7f239659c700 1 --2- 172.21.15.181:0/2232755265 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f23800b2990 0x7f23800b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:44.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.832+0000 7f23877fe700 1 -- 172.21.15.181:0/2232755265 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f23880e1a00 con 0x7f2398108400 2024-05-03T20:36:44.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.832+0000 7f239659c700 1 --2- 172.21.15.181:0/2232755265 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f23800b2990 0x7f23800b4e50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2390006fd0 tx=0x7f2390008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:44.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.833+0000 7f23a70c5b80 1 Processor -- start 2024-05-03T20:36:44.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.833+0000 7f23a70c5b80 1 -- start start 2024-05-03T20:36:44.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.833+0000 7f23a70c5b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0a8a15740 0x55c0a8a15b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:44.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.833+0000 7f23a70c5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0a8a16060 0x55c0a8a10f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:44.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.833+0000 7f23a70c5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a11480 0x55c0a8a118c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:44.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.833+0000 7f23a70c5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c0a89638f0 con 0x55c0a8a11480 2024-05-03T20:36:44.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.833+0000 7f23a70c5b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c0a88c8d50 con 0x55c0a8a15740 2024-05-03T20:36:44.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.833+0000 7f23a70c5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c0a895f8b0 con 0x55c0a8a16060 2024-05-03T20:36:44.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.833+0000 7f2396d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a11480 0x55c0a8a118c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:44.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.833+0000 7f2396d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a11480 0x55c0a8a118c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36568/0 (socket says 172.21.15.181:36568) 2024-05-03T20:36:44.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.833+0000 7f2396d9d700 1 -- 172.21.15.181:0/2016929629 learned_addr learned my addr 172.21.15.181:0/2016929629 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:44.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.833+0000 7f2395d9b700 1 --2- 172.21.15.181:0/2016929629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0a8a16060 0x55c0a8a10f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:44.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.833+0000 7f239659c700 1 --2- 172.21.15.181:0/2016929629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0a8a15740 0x55c0a8a15b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:44.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.833+0000 7f2396d9d700 1 -- 172.21.15.181:0/2016929629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0a8a16060 msgr2=0x55c0a8a10f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:44.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.833+0000 7f2396d9d700 1 --2- 172.21.15.181:0/2016929629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0a8a16060 0x55c0a8a10f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.833+0000 7f2396d9d700 1 -- 172.21.15.181:0/2016929629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0a8a15740 msgr2=0x55c0a8a15b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:44.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.833+0000 7f2396d9d700 1 --2- 172.21.15.181:0/2016929629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0a8a15740 0x55c0a8a15b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.833+0000 7f2396d9d700 1 -- 172.21.15.181:0/2016929629 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0a88b0880 con 0x55c0a8a11480 2024-05-03T20:36:44.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.834+0000 7f2396d9d700 1 --2- 172.21.15.181:0/2016929629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a11480 0x55c0a8a118c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f23880a5050 tx=0x7f23880107e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02823b8f5e4c05c server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:44.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.834+0000 7f2384ff9700 1 -- 172.21.15.181:0/2016929629 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23880a8ae0 con 0x55c0a8a11480 2024-05-03T20:36:44.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.834+0000 7f2384ff9700 1 -- 172.21.15.181:0/2016929629 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23880a96d0 con 0x55c0a8a11480 2024-05-03T20:36:44.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.834+0000 7f2384ff9700 1 -- 172.21.15.181:0/2016929629 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23880b19a0 con 0x55c0a8a11480 2024-05-03T20:36:44.838 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.834+0000 7f23a70c5b80 1 -- 172.21.15.181:0/2016929629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a11480 msgr2=0x55c0a8a118c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:44.838 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.834+0000 7f23a70c5b80 1 --2- 172.21.15.181:0/2016929629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a11480 0x55c0a8a118c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f23880a5050 tx=0x7f23880107e0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.838 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.835+0000 7f23a70c5b80 1 -- 172.21.15.181:0/2016929629 shutdown_connections 2024-05-03T20:36:44.838 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.835+0000 7f23a70c5b80 1 --2- 172.21.15.181:0/2016929629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a11480 0x55c0a8a118c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.838 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.835+0000 7f23a70c5b80 1 --2- 172.21.15.181:0/2016929629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0a8a15740 0x55c0a8a15b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.838 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.835+0000 7f23a70c5b80 1 --2- 172.21.15.181:0/2016929629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0a8a16060 0x55c0a8a10f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.838 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.835+0000 7f23a70c5b80 1 -- 172.21.15.181:0/2016929629 >> 172.21.15.181:0/2016929629 conn(0x55c0a893db20 msgr2=0x55c0a8a0b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:44.838 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.835+0000 7f23a70c5b80 1 -- 172.21.15.181:0/2016929629 shutdown_connections 2024-05-03T20:36:44.838 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.835+0000 7f23a70c5b80 1 -- 172.21.15.181:0/2016929629 wait complete. 2024-05-03T20:36:44.838 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.835+0000 7f23a70c5b80 1 -- start start 2024-05-03T20:36:44.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.835+0000 7f23a70c5b80 1 Processor -- start 2024-05-03T20:36:44.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.835+0000 7f23a70c5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a11480 0x55c0a8a1ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:44.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.835+0000 7f23a70c5b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0a8a15740 0x55c0a8a1a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:44.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.835+0000 7f23a70c5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0a8a16060 0x55c0a8a1a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:44.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.835+0000 7f23a70c5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c0a8a1dba0 con 0x55c0a8a11480 2024-05-03T20:36:44.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.835+0000 7f239659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a11480 0x55c0a8a1ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:44.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.835+0000 7f23a70c5b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c0a891ce80 con 0x55c0a8a15740 2024-05-03T20:36:44.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.835+0000 7f23a70c5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c0a8a1dd20 con 0x55c0a8a16060 2024-05-03T20:36:44.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.835+0000 7f239659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a11480 0x55c0a8a1ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36584/0 (socket says 172.21.15.181:36584) 2024-05-03T20:36:44.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.836+0000 7f239659c700 1 -- 172.21.15.181:0/3455732011 learned_addr learned my addr 172.21.15.181:0/3455732011 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:44.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.836+0000 7f23a70c5b80 10 client.0 _ll_register_callbacks cb 0x55c0a88f7560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T20:36:44.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.836+0000 7f2395d9b700 1 --2- 172.21.15.181:0/3455732011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0a8a15740 0x55c0a8a1a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:44.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.836+0000 7f2396d9d700 1 --2- 172.21.15.181:0/3455732011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0a8a16060 0x55c0a8a1a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:44.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.836+0000 7f239659c700 1 -- 172.21.15.181:0/3455732011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0a8a16060 msgr2=0x55c0a8a1a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:44.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.836+0000 7f239659c700 1 --2- 172.21.15.181:0/3455732011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0a8a16060 0x55c0a8a1a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.836+0000 7f239659c700 1 -- 172.21.15.181:0/3455732011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0a8a15740 msgr2=0x55c0a8a1a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:44.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.836+0000 7f239659c700 1 --2- 172.21.15.181:0/3455732011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0a8a15740 0x55c0a8a1a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.836+0000 7f239659c700 1 -- 172.21.15.181:0/3455732011 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0a891a710 con 0x55c0a8a11480 2024-05-03T20:36:44.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.836+0000 7f2396d9d700 1 --2- 172.21.15.181:0/3455732011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0a8a16060 0x55c0a8a1a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:36:44.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.836+0000 7f239659c700 1 --2- 172.21.15.181:0/3455732011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a11480 0x55c0a8a1ce30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f239000deb0 tx=0x7f2390004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be23285ef3ddd0c server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:44.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.836+0000 7f237dffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-03T20:36:44.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.836+0000 7f237dffb700 1 -- 172.21.15.181:0/3455732011 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2390024020 con 0x55c0a8a11480 2024-05-03T20:36:44.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.836+0000 7f237dffb700 1 -- 172.21.15.181:0/3455732011 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f239000f040 con 0x55c0a8a11480 2024-05-03T20:36:44.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.836+0000 7f23a70c5b80 1 -- 172.21.15.181:0/3455732011 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23880a96d0 con 0x55c0a8a11480 2024-05-03T20:36:44.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.836+0000 7f237dffb700 1 -- 172.21.15.181:0/3455732011 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2390014e60 con 0x55c0a8a11480 2024-05-03T20:36:44.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.836+0000 7f23a70c5b80 10 client.15150 fetch_fsmap learned FSMap version 27 2024-05-03T20:36:44.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.836+0000 7f23a70c5b80 1 -- 172.21.15.181:0/3455732011 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c0a8a28440 con 0x55c0a8a11480 2024-05-03T20:36:44.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.836+0000 7f237dffb700 1 -- 172.21.15.181:0/3455732011 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23880a96d0 con 0x55c0a8a11480 2024-05-03T20:36:44.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.837+0000 7f237dffb700 1 -- 172.21.15.181:0/3455732011 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2390013070 con 0x55c0a8a11480 2024-05-03T20:36:44.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.837+0000 7f23a70c5b80 10 client.15150 fetch_fsmap finished waiting for FSMap version 27 2024-05-03T20:36:44.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.837+0000 7f23a70c5b80 10 client.15150 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-03T20:36:44.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.837+0000 7f23a70c5b80 20 client.15150 populate_metadata read hostname 'smithi181' 2024-05-03T20:36:44.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.837+0000 7f23a70c5b80 1 --2- 172.21.15.181:0/3455732011 >> [v2:172.21.15.196:6838/1182826743,v1:172.21.15.196:6839/1182826743] conn(0x55c0a8a2aae0 0x55c0a8a2bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:44.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.837+0000 7f23a70c5b80 4 client.15150 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T20:36:44.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.837+0000 7f23a70c5b80 1 -- 172.21.15.181:0/3455732011 --> [v2:172.21.15.196:6838/1182826743,v1:172.21.15.196:6839/1182826743] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0a89712d0 con 0x55c0a8a2aae0 2024-05-03T20:36:44.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.837+0000 7f2395d9b700 1 --2- 172.21.15.181:0/3455732011 >> [v2:172.21.15.196:6838/1182826743,v1:172.21.15.196:6839/1182826743] conn(0x55c0a8a2aae0 0x55c0a8a2bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:44.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.837+0000 7f2395d9b700 1 --2- 172.21.15.181:0/3455732011 >> [v2:172.21.15.196:6838/1182826743,v1:172.21.15.196:6839/1182826743] conn(0x55c0a8a2aae0 0x55c0a8a2bf50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50c0b7fef5de0624 server_cookie=f352cfe5b6569636 in_seq=0 out_seq=0 2024-05-03T20:36:44.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.837+0000 7f237dffb700 10 client.15150 ms_handle_connect on v2:172.21.15.196:6838/1182826743 2024-05-03T20:36:44.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.838+0000 7f237dffb700 1 -- 172.21.15.181:0/3455732011 <== mds.0 v2:172.21.15.196:6838/1182826743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c0a89712d0 con 0x55c0a8a2aae0 2024-05-03T20:36:44.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.838+0000 7f237dffb700 10 client.15150 handle_command_reply: tid=0 2024-05-03T20:36:44.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.838+0000 7f23a70c5b80 1 client.15150 shutdown 2024-05-03T20:36:44.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.838+0000 7f23a70c5b80 1 -- 172.21.15.181:0/3455732011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a11480 msgr2=0x55c0a8a1ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:44.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.838+0000 7f23a70c5b80 1 --2- 172.21.15.181:0/3455732011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a11480 0x55c0a8a1ce30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f239000deb0 tx=0x7f2390004860 comp rx=0 tx=0).stop 2024-05-03T20:36:44.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.838+0000 7f23a70c5b80 1 -- 172.21.15.181:0/3455732011 shutdown_connections 2024-05-03T20:36:44.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.838+0000 7f23a70c5b80 1 --2- 172.21.15.181:0/3455732011 >> [v2:172.21.15.196:6838/1182826743,v1:172.21.15.196:6839/1182826743] conn(0x55c0a8a2aae0 0x55c0a8a2bf50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.838+0000 7f23a70c5b80 1 --2- 172.21.15.181:0/3455732011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0a8a16060 0x55c0a8a1a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.838+0000 7f237dffb700 0 client.15150 ms_handle_reset on v2:172.21.15.196:6838/1182826743 2024-05-03T20:36:44.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.838+0000 7f23a70c5b80 1 --2- 172.21.15.181:0/3455732011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a11480 0x55c0a8a1ce30 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.838+0000 7f23a70c5b80 1 --2- 172.21.15.181:0/3455732011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0a8a15740 0x55c0a8a1a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.838+0000 7f23a70c5b80 1 -- 172.21.15.181:0/3455732011 >> 172.21.15.181:0/3455732011 conn(0x55c0a893db20 msgr2=0x55c0a8a16d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:44.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.838+0000 7f23a70c5b80 1 -- 172.21.15.181:0/3455732011 shutdown_connections 2024-05-03T20:36:44.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.838+0000 7f23a70c5b80 1 -- 172.21.15.181:0/3455732011 wait complete. 2024-05-03T20:36:44.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.838+0000 7f23a70c5b80 20 client.15150 trim_cache size 0 max 16384 2024-05-03T20:36:44.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.859+0000 7f23a70c5b80 1 Processor -- start 2024-05-03T20:36:44.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.859+0000 7f23a70c5b80 1 -- start start 2024-05-03T20:36:44.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.859+0000 7f23a70c5b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0a8a11480 0x55c0a8a1c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:44.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.859+0000 7f23a70c5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a15740 0x55c0a8a0ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:44.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.859+0000 7f23a70c5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0a8a16060 0x55c0a8a0f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:44.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.859+0000 7f23a70c5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c0a8a2a290 con 0x55c0a8a15740 2024-05-03T20:36:44.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.859+0000 7f23a70c5b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c0a8a2a960 con 0x55c0a8a11480 2024-05-03T20:36:44.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.859+0000 7f23a70c5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c0a8a2b030 con 0x55c0a8a16060 2024-05-03T20:36:44.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.859+0000 7f2395d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a15740 0x55c0a8a0ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:44.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.859+0000 7f2395d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a15740 0x55c0a8a0ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36598/0 (socket says 172.21.15.181:36598) 2024-05-03T20:36:44.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.859+0000 7f2395d9b700 1 -- 172.21.15.181:0/2699956626 learned_addr learned my addr 172.21.15.181:0/2699956626 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:44.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.859+0000 7f239659c700 1 --2- 172.21.15.181:0/2699956626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0a8a11480 0x55c0a8a1c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:44.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.859+0000 7f2396d9d700 1 --2- 172.21.15.181:0/2699956626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0a8a16060 0x55c0a8a0f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:44.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.859+0000 7f2395d9b700 1 -- 172.21.15.181:0/2699956626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0a8a16060 msgr2=0x55c0a8a0f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:44.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.859+0000 7f2395d9b700 1 --2- 172.21.15.181:0/2699956626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0a8a16060 0x55c0a8a0f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.859+0000 7f2395d9b700 1 -- 172.21.15.181:0/2699956626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0a8a11480 msgr2=0x55c0a8a1c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:44.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.859+0000 7f2395d9b700 1 --2- 172.21.15.181:0/2699956626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0a8a11480 0x55c0a8a1c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.859+0000 7f2395d9b700 1 -- 172.21.15.181:0/2699956626 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0a8a18740 con 0x55c0a8a15740 2024-05-03T20:36:44.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.860+0000 7f2396d9d700 1 --2- 172.21.15.181:0/2699956626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0a8a16060 0x55c0a8a0f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:36:44.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.860+0000 7f2395d9b700 1 --2- 172.21.15.181:0/2699956626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a15740 0x55c0a8a0ef30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f238c015000 tx=0x7f238c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f51610112659ec server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:44.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.860+0000 7f237ffff700 1 -- 172.21.15.181:0/2699956626 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c004c80 con 0x55c0a8a15740 2024-05-03T20:36:44.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.860+0000 7f237ffff700 1 -- 172.21.15.181:0/2699956626 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f238c004e20 con 0x55c0a8a15740 2024-05-03T20:36:44.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.860+0000 7f237ffff700 1 -- 172.21.15.181:0/2699956626 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c0079c0 con 0x55c0a8a15740 2024-05-03T20:36:44.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.860+0000 7f23a70c5b80 1 -- 172.21.15.181:0/2699956626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a15740 msgr2=0x55c0a8a0ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:44.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.860+0000 7f23a70c5b80 1 --2- 172.21.15.181:0/2699956626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a15740 0x55c0a8a0ef30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f238c015000 tx=0x7f238c009c20 comp rx=0 tx=0).stop 2024-05-03T20:36:44.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.861+0000 7f23a70c5b80 1 -- 172.21.15.181:0/2699956626 shutdown_connections 2024-05-03T20:36:44.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.861+0000 7f23a70c5b80 1 --2- 172.21.15.181:0/2699956626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0a8a16060 0x55c0a8a0f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.861+0000 7f23a70c5b80 1 --2- 172.21.15.181:0/2699956626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0a8a11480 0x55c0a8a1c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.861+0000 7f23a70c5b80 1 --2- 172.21.15.181:0/2699956626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a15740 0x55c0a8a0ef30 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.861+0000 7f23a70c5b80 1 -- 172.21.15.181:0/2699956626 >> 172.21.15.181:0/2699956626 conn(0x55c0a893db20 msgr2=0x55c0a8a1a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:44.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.861+0000 7f23a70c5b80 1 -- 172.21.15.181:0/2699956626 shutdown_connections 2024-05-03T20:36:44.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.861+0000 7f23a70c5b80 1 -- 172.21.15.181:0/2699956626 wait complete. 2024-05-03T20:36:44.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.861+0000 7f23a70c5b80 1 -- start start 2024-05-03T20:36:44.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.861+0000 7f23a70c5b80 1 Processor -- start 2024-05-03T20:36:44.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.861+0000 7f23a70c5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a11480 0x55c0a8a353d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:44.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.861+0000 7f23a70c5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0a8a15740 0x55c0a8a35910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:44.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.861+0000 7f23a70c5b80 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0a8a16060 0x55c0a8a2e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:44.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.861+0000 7f23a70c5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c0a8a2bdd0 con 0x55c0a8a11480 2024-05-03T20:36:44.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.861+0000 7f23a70c5b80 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c0a8a2b700 con 0x55c0a8a16060 2024-05-03T20:36:44.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.861+0000 7f23a70c5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c0a8a2c4a0 con 0x55c0a8a15740 2024-05-03T20:36:44.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.861+0000 7f23a70c5b80 10 client.0 _ll_register_callbacks cb 0x55c0a88f7560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T20:36:44.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.861+0000 7f239659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a11480 0x55c0a8a353d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:44.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.861+0000 7f239659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a11480 0x55c0a8a353d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36606/0 (socket says 172.21.15.181:36606) 2024-05-03T20:36:44.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.861+0000 7f239659c700 1 -- 172.21.15.181:0/80167121 learned_addr learned my addr 172.21.15.181:0/80167121 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:44.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.861+0000 7f2395d9b700 1 --2- 172.21.15.181:0/80167121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0a8a15740 0x55c0a8a35910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:44.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.861+0000 7f2396d9d700 1 --2- 172.21.15.181:0/80167121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0a8a16060 0x55c0a8a2e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:44.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.862+0000 7f239659c700 1 -- 172.21.15.181:0/80167121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0a8a15740 msgr2=0x55c0a8a35910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:44.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.862+0000 7f239659c700 1 --2- 172.21.15.181:0/80167121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0a8a15740 0x55c0a8a35910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.862+0000 7f239659c700 1 -- 172.21.15.181:0/80167121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0a8a16060 msgr2=0x55c0a8a2e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T20:36:44.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.862+0000 7f239659c700 1 --2- 172.21.15.181:0/80167121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0a8a16060 0x55c0a8a2e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.862+0000 7f239659c700 1 -- 172.21.15.181:0/80167121 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0a8a2ebb0 con 0x55c0a8a11480 2024-05-03T20:36:44.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.862+0000 7f2396d9d700 1 --2- 172.21.15.181:0/80167121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0a8a16060 0x55c0a8a2e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:36:44.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.862+0000 7f239659c700 1 --2- 172.21.15.181:0/80167121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a11480 0x55c0a8a353d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2390004790 tx=0x7f239002bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a484441cbd18fbae server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:44.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.862+0000 7f237dffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-03T20:36:44.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.862+0000 7f237dffb700 1 -- 172.21.15.181:0/80167121 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2390039060 con 0x55c0a8a11480 2024-05-03T20:36:44.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.862+0000 7f237dffb700 1 -- 172.21.15.181:0/80167121 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f23900209a0 con 0x55c0a8a11480 2024-05-03T20:36:44.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.862+0000 7f23a70c5b80 1 -- 172.21.15.181:0/80167121 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f238c004e20 con 0x55c0a8a11480 2024-05-03T20:36:44.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.862+0000 7f237dffb700 1 -- 172.21.15.181:0/80167121 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2390024040 con 0x55c0a8a11480 2024-05-03T20:36:44.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.862+0000 7f23a70c5b80 10 client.15156 fetch_fsmap learned FSMap version 27 2024-05-03T20:36:44.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.862+0000 7f23a70c5b80 1 -- 172.21.15.181:0/80167121 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c0a8a96a00 con 0x55c0a8a11480 2024-05-03T20:36:44.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.862+0000 7f237dffb700 1 -- 172.21.15.181:0/80167121 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f238c004e20 con 0x55c0a8a11480 2024-05-03T20:36:44.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.863+0000 7f237dffb700 1 -- 172.21.15.181:0/80167121 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2390006020 con 0x55c0a8a11480 2024-05-03T20:36:44.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.863+0000 7f23a70c5b80 10 client.15156 fetch_fsmap finished waiting for FSMap version 27 2024-05-03T20:36:44.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.863+0000 7f23a70c5b80 10 client.15156 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-03T20:36:44.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.863+0000 7f23a70c5b80 20 client.15156 populate_metadata read hostname 'smithi181' 2024-05-03T20:36:44.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.863+0000 7f23a70c5b80 1 --2- 172.21.15.181:0/80167121 >> [v2:172.21.15.196:6838/1182826743,v1:172.21.15.196:6839/1182826743] conn(0x55c0a8a2aae0 0x55c0a8a9a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:44.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.863+0000 7f23a70c5b80 4 client.15156 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T20:36:44.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.863+0000 7f23a70c5b80 1 -- 172.21.15.181:0/80167121 --> [v2:172.21.15.196:6838/1182826743,v1:172.21.15.196:6839/1182826743] -- command(tid 0: {"prefix": "status"}) -- 0x55c0a896b370 con 0x55c0a8a2aae0 2024-05-03T20:36:44.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.863+0000 7f2395d9b700 1 --2- 172.21.15.181:0/80167121 >> [v2:172.21.15.196:6838/1182826743,v1:172.21.15.196:6839/1182826743] conn(0x55c0a8a2aae0 0x55c0a8a9a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:44.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.863+0000 7f237dffb700 10 client.15156 ms_handle_connect on v2:172.21.15.196:6838/1182826743 2024-05-03T20:36:44.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.863+0000 7f2395d9b700 1 --2- 172.21.15.181:0/80167121 >> [v2:172.21.15.196:6838/1182826743,v1:172.21.15.196:6839/1182826743] conn(0x55c0a8a2aae0 0x55c0a8a9a370 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=938e97199e8c1cc5 server_cookie=5e64ef156f868423 in_seq=0 out_seq=0 2024-05-03T20:36:44.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.863+0000 7f237dffb700 1 -- 172.21.15.181:0/80167121 <== mds.0 v2:172.21.15.196:6838/1182826743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c0a896b370 con 0x55c0a8a2aae0 2024-05-03T20:36:44.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.863+0000 7f237dffb700 10 client.15156 handle_command_reply: tid=0 2024-05-03T20:36:44.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.864+0000 7f23a70c5b80 1 client.15156 shutdown 2024-05-03T20:36:44.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.864+0000 7f23a70c5b80 1 -- 172.21.15.181:0/80167121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a11480 msgr2=0x55c0a8a353d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:44.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.864+0000 7f23a70c5b80 1 --2- 172.21.15.181:0/80167121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a11480 0x55c0a8a353d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2390004790 tx=0x7f239002bbe0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.864+0000 7f23a70c5b80 1 -- 172.21.15.181:0/80167121 shutdown_connections 2024-05-03T20:36:44.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.864+0000 7f23a70c5b80 1 --2- 172.21.15.181:0/80167121 >> [v2:172.21.15.196:6838/1182826743,v1:172.21.15.196:6839/1182826743] conn(0x55c0a8a2aae0 0x55c0a8a9a370 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.864+0000 7f23a70c5b80 1 --2- 172.21.15.181:0/80167121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c0a8a16060 0x55c0a8a2e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.864+0000 7f23a70c5b80 1 --2- 172.21.15.181:0/80167121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c0a8a11480 0x55c0a8a353d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.864+0000 7f237dffb700 0 client.15156 ms_handle_reset on v2:172.21.15.196:6838/1182826743 2024-05-03T20:36:44.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.864+0000 7f23a70c5b80 1 --2- 172.21.15.181:0/80167121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0a8a15740 0x55c0a8a35910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.864+0000 7f23a70c5b80 1 -- 172.21.15.181:0/80167121 >> 172.21.15.181:0/80167121 conn(0x55c0a893db20 msgr2=0x55c0a8a19350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:44.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.864+0000 7f23a70c5b80 1 -- 172.21.15.181:0/80167121 shutdown_connections 2024-05-03T20:36:44.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.864+0000 7f23a70c5b80 1 -- 172.21.15.181:0/80167121 wait complete. 2024-05-03T20:36:44.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.864+0000 7f23a70c5b80 20 client.15156 trim_cache size 0 max 16384 2024-05-03T20:36:44.868 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-05-03T20:36:44.868 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "1cd058ee-098b-11ef-bc97-c7b262605968", 2024-05-03T20:36:44.868 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 0, 2024-05-03T20:36:44.869 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 24488, 2024-05-03T20:36:44.869 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2024-05-03T20:36:44.869 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2024-05-03T20:36:44.869 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2024-05-03T20:36:44.869 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 182.15938077300001, 2024-05-03T20:36:44.869 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 27, 2024-05-03T20:36:44.869 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 83, 2024-05-03T20:36:44.869 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 81, 2024-05-03T20:36:44.869 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 196.65968631199999 2024-05-03T20:36:44.869 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-05-03T20:36:44.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.864+0000 7f239c832700 1 -- 172.21.15.181:0/2232755265 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f23800b2990 msgr2=0x7f23800b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:44.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.864+0000 7f239c832700 1 --2- 172.21.15.181:0/2232755265 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f23800b2990 0x7f23800b4e50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2390006fd0 tx=0x7f2390008040 comp rx=0 tx=0).stop 2024-05-03T20:36:44.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.864+0000 7f239c832700 1 -- 172.21.15.181:0/2232755265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2398108400 msgr2=0x7f23981105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:44.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.864+0000 7f239c832700 1 --2- 172.21.15.181:0/2232755265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2398108400 0x7f23981105d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f238800b550 tx=0x7f238800b910 comp rx=0 tx=0).stop 2024-05-03T20:36:44.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.864+0000 7f239c832700 1 -- 172.21.15.181:0/2232755265 shutdown_connections 2024-05-03T20:36:44.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.864+0000 7f239c832700 1 --2- 172.21.15.181:0/2232755265 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f23800b2990 0x7f23800b4e50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.864+0000 7f239c832700 1 --2- 172.21.15.181:0/2232755265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2398108400 0x7f23981105d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.864+0000 7f239c832700 1 --2- 172.21.15.181:0/2232755265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2398102390 0x7f239810fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.864+0000 7f239c832700 1 --2- 172.21.15.181:0/2232755265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2398102d50 0x7f2398110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:44.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.864+0000 7f239c832700 1 -- 172.21.15.181:0/2232755265 >> 172.21.15.181:0/2232755265 conn(0x7f2398079310 msgr2=0x7f2398101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:44.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.865+0000 7f239c832700 1 -- 172.21.15.181:0/2232755265 shutdown_connections 2024-05-03T20:36:44.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:44.865+0000 7f239c832700 1 -- 172.21.15.181:0/2232755265 wait complete. 2024-05-03T20:36:45.590 INFO:teuthology.run_tasks:Running task sequential... 2024-05-03T20:36:45.599 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-03T20:36:45.600 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-03T20:36:45.600 INFO:teuthology.task.print:Enabling mgr modules 2024-05-03T20:36:45.600 INFO:teuthology.run_tasks:Running task kclient... 2024-05-03T20:36:45.656 INFO:tasks.kclient:Mounting kernel clients... 2024-05-03T20:36:45.657 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-03T20:36:45.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T20:36:45.657 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-05-03T20:36:45.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T20:36:45.673 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link delete ceph-brx 2024-05-03T20:36:45.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:45 smithi181 ceph-mon[29801]: pgmap v378: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-05-03T20:36:45.740 INFO:teuthology.orchestra.run.smithi181.stderr:Cannot find device "ceph-brx" 2024-05-03T20:36:45.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T20:36:45.741 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-05-03T20:36:45.741 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-03T20:36:45.742 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-03T20:36:45.742 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-03T20:36:45.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:45 smithi196 ceph-mon[32858]: pgmap v378: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-05-03T20:36:45.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:45 smithi204 ceph-mon[33595]: pgmap v378: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-05-03T20:36:46.041 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:36:47.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.675+0000 7f34bc6cb700 1 -- 172.21.15.181:0/1425987778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34b4106f40 msgr2=0x7f34b410ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:47.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.675+0000 7f34bc6cb700 1 --2- 172.21.15.181:0/1425987778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34b4106f40 0x7f34b410ea80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00b0a0 tx=0x7f34ac00b3b0 comp rx=0 tx=0).stop 2024-05-03T20:36:47.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.676+0000 7f34bc6cb700 1 -- 172.21.15.181:0/1425987778 shutdown_connections 2024-05-03T20:36:47.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.676+0000 7f34bc6cb700 1 --2- 172.21.15.181:0/1425987778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34b4106f40 0x7f34b410ea80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:47.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.676+0000 7f34bc6cb700 1 --2- 172.21.15.181:0/1425987778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34b41020a0 0x7f34b4102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:47.675 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.676+0000 7f34bc6cb700 1 --2- 172.21.15.181:0/1425987778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34b4102a50 0x7f34b4106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:47.675 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.676+0000 7f34bc6cb700 1 -- 172.21.15.181:0/1425987778 >> 172.21.15.181:0/1425987778 conn(0x7f34b40776c0 msgr2=0x7f34b4077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:47.675 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:47 smithi181 ceph-mon[29801]: pgmap v379: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T20:36:47.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.676+0000 7f34bc6cb700 1 -- 172.21.15.181:0/1425987778 shutdown_connections 2024-05-03T20:36:47.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.676+0000 7f34bc6cb700 1 -- 172.21.15.181:0/1425987778 wait complete. 2024-05-03T20:36:47.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.676+0000 7f34bc6cb700 1 Processor -- start 2024-05-03T20:36:47.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.677+0000 7f34bc6cb700 1 -- start start 2024-05-03T20:36:47.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.677+0000 7f34bc6cb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34b41020a0 0x7f34b4112710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:47.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.677+0000 7f34bc6cb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34b4102a50 0x7f34b4112c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:47.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.677+0000 7f34bc6cb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34b4106f40 0x7f34b419e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:47.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.677+0000 7f34bc6cb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f34b4111d80 con 0x7f34b41020a0 2024-05-03T20:36:47.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.677+0000 7f34bc6cb700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f34b4111c00 con 0x7f34b4102a50 2024-05-03T20:36:47.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.677+0000 7f34bc6cb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f34b4111f00 con 0x7f34b4106f40 2024-05-03T20:36:47.677 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.677+0000 7f34ba467700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34b41020a0 0x7f34b4112710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:47.678 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.677+0000 7f34ba467700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34b41020a0 0x7f34b4112710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36538/0 (socket says 172.21.15.181:36538) 2024-05-03T20:36:47.678 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.677+0000 7f34ba467700 1 -- 172.21.15.181:0/4049759653 learned_addr learned my addr 172.21.15.181:0/4049759653 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:47.678 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.677+0000 7f34bac68700 1 --2- 172.21.15.181:0/4049759653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34b4106f40 0x7f34b419e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:47.678 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.677+0000 7f34b9c66700 1 --2- 172.21.15.181:0/4049759653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34b4102a50 0x7f34b4112c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:47.678 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.677+0000 7f34ba467700 1 -- 172.21.15.181:0/4049759653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34b4106f40 msgr2=0x7f34b419e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:47.678 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.677+0000 7f34ba467700 1 --2- 172.21.15.181:0/4049759653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34b4106f40 0x7f34b419e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:47.678 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.677+0000 7f34ba467700 1 -- 172.21.15.181:0/4049759653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34b4102a50 msgr2=0x7f34b4112c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:47.678 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.677+0000 7f34ba467700 1 --2- 172.21.15.181:0/4049759653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34b4102a50 0x7f34b4112c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:47.678 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.677+0000 7f34ba467700 1 -- 172.21.15.181:0/4049759653 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b419e9b0 con 0x7f34b41020a0 2024-05-03T20:36:47.678 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.677+0000 7f34b9c66700 1 --2- 172.21.15.181:0/4049759653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34b4102a50 0x7f34b4112c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:36:47.678 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.677+0000 7f34ba467700 1 --2- 172.21.15.181:0/4049759653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34b41020a0 0x7f34b4112710 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f34b000c9b0 tx=0x7f34b000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:47.678 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.678+0000 7f34a77fe700 1 -- 172.21.15.181:0/4049759653 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b0007b00 con 0x7f34b41020a0 2024-05-03T20:36:47.678 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.678+0000 7f34a77fe700 1 -- 172.21.15.181:0/4049759653 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34b0007ca0 con 0x7f34b41020a0 2024-05-03T20:36:47.678 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.678+0000 7f34bc6cb700 1 -- 172.21.15.181:0/4049759653 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b419eca0 con 0x7f34b41020a0 2024-05-03T20:36:47.679 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.678+0000 7f34a77fe700 1 -- 172.21.15.181:0/4049759653 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b0018660 con 0x7f34b41020a0 2024-05-03T20:36:47.679 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.678+0000 7f34bc6cb700 1 -- 172.21.15.181:0/4049759653 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b41a6590 con 0x7f34b41020a0 2024-05-03T20:36:47.679 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.679+0000 7f34bc6cb700 1 -- 172.21.15.181:0/4049759653 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b406a190 con 0x7f34b41020a0 2024-05-03T20:36:47.679 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.680+0000 7f34a77fe700 1 -- 172.21.15.181:0/4049759653 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f34b000f450 con 0x7f34b41020a0 2024-05-03T20:36:47.679 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.681+0000 7f34a77fe700 1 --2- 172.21.15.181:0/4049759653 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f34a00b2910 0x7f34a00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:47.679 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.681+0000 7f34b9c66700 1 --2- 172.21.15.181:0/4049759653 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f34a00b2910 0x7f34a00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:47.680 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.681+0000 7f34a77fe700 1 -- 172.21.15.181:0/4049759653 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f34b00e2630 con 0x7f34b41020a0 2024-05-03T20:36:47.680 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.681+0000 7f34b9c66700 1 --2- 172.21.15.181:0/4049759653 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f34a00b2910 0x7f34a00b4dd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f34a8005fd0 tx=0x7f34a8005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:47.683 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.684+0000 7f34a77fe700 1 -- 172.21.15.181:0/4049759653 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f34b0014030 con 0x7f34b41020a0 2024-05-03T20:36:47.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:47 smithi196 ceph-mon[32858]: pgmap v379: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T20:36:47.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:47 smithi204 ceph-mon[33595]: pgmap v379: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T20:36:47.932 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.933+0000 7f34bc6cb700 1 -- 172.21.15.181:0/4049759653 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f34b419ad50 con 0x7f34b41020a0 2024-05-03T20:36:47.932 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.934+0000 7f34a77fe700 1 -- 172.21.15.181:0/4049759653 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f34b00aa280 con 0x7f34b41020a0 2024-05-03T20:36:47.932 INFO:teuthology.orchestra.run.smithi181.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-03T20:36:47.934 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.936+0000 7f34bc6cb700 1 -- 172.21.15.181:0/4049759653 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f34a00b2910 msgr2=0x7f34a00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:47.935 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.936+0000 7f34bc6cb700 1 --2- 172.21.15.181:0/4049759653 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f34a00b2910 0x7f34a00b4dd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f34a8005fd0 tx=0x7f34a8005e20 comp rx=0 tx=0).stop 2024-05-03T20:36:47.935 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.936+0000 7f34bc6cb700 1 -- 172.21.15.181:0/4049759653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34b41020a0 msgr2=0x7f34b4112710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:47.935 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.936+0000 7f34bc6cb700 1 --2- 172.21.15.181:0/4049759653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34b41020a0 0x7f34b4112710 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f34b000c9b0 tx=0x7f34b000ccc0 comp rx=0 tx=0).stop 2024-05-03T20:36:47.935 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.936+0000 7f34bc6cb700 1 -- 172.21.15.181:0/4049759653 shutdown_connections 2024-05-03T20:36:47.935 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.936+0000 7f34bc6cb700 1 --2- 172.21.15.181:0/4049759653 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f34a00b2910 0x7f34a00b4dd0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:47.935 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.936+0000 7f34bc6cb700 1 --2- 172.21.15.181:0/4049759653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34b4106f40 0x7f34b419e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:47.935 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.936+0000 7f34bc6cb700 1 --2- 172.21.15.181:0/4049759653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34b41020a0 0x7f34b4112710 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:47.935 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.936+0000 7f34bc6cb700 1 --2- 172.21.15.181:0/4049759653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34b4102a50 0x7f34b4112c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:47.935 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.936+0000 7f34bc6cb700 1 -- 172.21.15.181:0/4049759653 >> 172.21.15.181:0/4049759653 conn(0x7f34b40776c0 msgr2=0x7f34b4101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:47.935 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.936+0000 7f34bc6cb700 1 -- 172.21.15.181:0/4049759653 shutdown_connections 2024-05-03T20:36:47.935 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:47.936+0000 7f34bc6cb700 1 -- 172.21.15.181:0/4049759653 wait complete. 2024-05-03T20:36:48.619 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:48 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/4049759653' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-03T20:36:48.648 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-03T20:36:48.649 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-03T20:36:48.649 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2024-05-03T20:36:48.649 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-03T20:36:48.649 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-03T20:36:48.649 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-03T20:36:48.649 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-03T20:36:48.649 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-03T20:36:48.649 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-03T20:36:48.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T20:36:48.650 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2024-05-03T20:36:48.666 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-03T20:36:48.666 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-03T20:36:48.666 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2024-05-03T20:36:48.666 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-05-03T20:36:48.667 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2024-05-03T20:36:48.667 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-05-03T20:36:48.667 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-03T20:36:48.667 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2024-05-03T20:36:48.667 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2024-05-03T20:36:48.667 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-03T20:36:48.667 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2024-05-03T20:36:48.667 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2024-05-03T20:36:48.668 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-03T20:36:48.668 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2024-05-03T20:36:48.668 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-03T20:36:48.668 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2024-05-03T20:36:48.668 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-03T20:36:48.668 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 41888sec preferred_lft 41888sec 2024-05-03T20:36:48.668 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::8c6:cae9:edfa:417a/64 scope link noprefixroute 2024-05-03T20:36:48.668 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-05-03T20:36:48.668 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-03T20:36:48.669 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T20:36:48.669 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-05-03T20:36:48.669 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link add name ceph-brx type bridge 2024-05-03T20:36:48.669 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip addr flush dev ceph-brx 2024-05-03T20:36:48.669 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set ceph-brx up 2024-05-03T20:36:48.669 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-03T20:36:48.669 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-05-03T20:36:48.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T20:36:48.847 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T20:36:48.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T20:36:48.854 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-03T20:36:48.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:48 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/4049759653' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-03T20:36:48.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:48 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/4049759653' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-03T20:36:48.878 INFO:teuthology.orchestra.run.smithi181.stdout:1 2024-05-03T20:36:48.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T20:36:48.880 DEBUG:teuthology.orchestra.run.smithi181:> route 2024-05-03T20:36:48.944 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2024-05-03T20:36:48.944 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-03T20:36:48.944 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-03T20:36:48.944 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-03T20:36:48.944 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-03T20:36:48.945 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T20:36:48.945 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-05-03T20:36:48.945 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-03T20:36:48.945 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-03T20:36:48.945 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-03T20:36:48.945 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-05-03T20:36:49.033 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T20:36:49.198 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T20:36:49.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T20:36:49.204 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-05-03T20:36:49.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T20:36:49.220 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list-id 2024-05-03T20:36:49.274 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T20:36:49.275 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-05-03T20:36:49.275 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-03T20:36:49.275 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-03T20:36:49.275 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-05-03T20:36:49.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T20:36:49.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T20:36:49.390 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-03T20:36:49.390 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T20:36:49.390 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-05-03T20:36:49.390 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-03T20:36:49.390 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-03T20:36:49.390 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-03T20:36:49.390 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-03T20:36:49.391 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-03T20:36:49.391 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-05-03T20:36:49.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T20:36:49.526 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T20:36:49.533 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T20:36:49.533 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-05-03T20:36:49.533 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2024-05-03T20:36:49.533 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set dev brx.0 master ceph-brx 2024-05-03T20:36:49.533 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-05-03T20:36:49.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T20:36:49.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T20:36:49.607 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-03T20:36:49.607 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-03T20:36:49.622 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-03T20:36:49.622 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-03T20:36:49.622 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-03T20:36:49.673 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-03T20:36:49.674 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-03T20:36:49.674 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/541f494e-3959-4449-bf1a-b4e2dce12c39 2024-05-03T20:36:49.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T20:36:49.674 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/541f494e-3959-4449-bf1a-b4e2dce12c39 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-05-03T20:36:49.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:49 smithi181 ceph-mon[29801]: pgmap v380: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T20:36:49.763 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-05-03T20:36:49.763 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-05-03T20:36:49.763 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2024-05-03T20:36:49.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:49 smithi196 ceph-mon[32858]: pgmap v380: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T20:36:49.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:49 smithi204 ceph-mon[33595]: pgmap v380: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T20:36:49.869 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-05-03T20:36:49.869 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-05-03T20:36:49.869 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2024-05-03T20:36:49.870 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.196:3300,172.21.15.204:3300" 2024-05-03T20:36:49.870 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.181:3300,172.21.15.196:3300,172.21.15.204:3300:/volumes/qa/sv_1/541f494e-3959-4449-bf1a-b4e2dce12c39 2024-05-03T20:36:49.870 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=1cd058ee-098b-11ef-bc97-c7b262605968" will pass to kernel 2024-05-03T20:36:49.871 INFO:teuthology.orchestra.run.smithi181.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-03T20:36:49.871 INFO:teuthology.orchestra.run.smithi181.stdout: You just mounted a file system that supports labels which does not 2024-05-03T20:36:49.871 INFO:teuthology.orchestra.run.smithi181.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-03T20:36:49.871 INFO:teuthology.orchestra.run.smithi181.stdout: applications will generate AVC messages and not be allowed access to 2024-05-03T20:36:49.871 INFO:teuthology.orchestra.run.smithi181.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-03T20:36:49.872 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-03T20:36:49.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T20:36:49.873 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-03T20:36:49.901 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-03T20:36:49.990 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-03T20:36:49.992 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T20:36:49.998 INFO:teuthology.orchestra.run.smithi181.stdout:client24815 2024-05-03T20:36:49.998 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-03T20:36:49.998 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-03T20:36:50.015 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-03T20:36:50.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-03T20:36:50.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T20:36:50.110 INFO:teuthology.orchestra.run.smithi181.stdout:client24815 2024-05-03T20:36:50.110 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-03T20:36:50.158 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-03T20:36:50.159 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T20:36:50.165 INFO:teuthology.orchestra.run.smithi181.stdout:client24815 2024-05-03T20:36:50.165 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/1cd058ee-098b-11ef-bc97-c7b262605968.client24815/status') 2024-05-03T20:36:50.215 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T20:36:50.224 INFO:teuthology.orchestra.run.smithi181.stdout:instance: client.24815 (3)172.21.15.181:0/1506339975 2024-05-03T20:36:50.224 INFO:teuthology.orchestra.run.smithi181.stdout:blocklisted: false 2024-05-03T20:36:50.224 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2024-05-03T20:36:50.224 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2024-05-03T20:36:50.224 INFO:teuthology.orchestra.run.smithi181.stderr:72 bytes copied, 2.2704e-05 s, 3.2 MB/s 2024-05-03T20:36:50.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T20:36:50.232 INFO:teuthology.run_tasks:Running task exec... 2024-05-03T20:36:50.245 INFO:teuthology.task.exec:Executing custom commands... 2024-05-03T20:36:50.245 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi181.front.sepia.ceph.com 2024-05-03T20:36:50.246 DEBUG:teuthology.orchestra.run.smithi181:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-05-03T20:36:50.352 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.351+0000 7f8ef1872640 1 -- 172.21.15.181:0/1305532764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8eec105ea0 msgr2=0x7f8eec108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:50.352 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.351+0000 7f8ef1872640 1 --2- 172.21.15.181:0/1305532764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8eec105ea0 0x7f8eec108290 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0009900 tx=0x7f8ee002a410 comp rx=0 tx=0).stop 2024-05-03T20:36:50.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.351+0000 7f8ef1872640 1 -- 172.21.15.181:0/1305532764 shutdown_connections 2024-05-03T20:36:50.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.351+0000 7f8ef1872640 1 --2- 172.21.15.181:0/1305532764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8eec105ea0 0x7f8eec108290 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:50.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.351+0000 7f8ef1872640 1 --2- 172.21.15.181:0/1305532764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eec1087d0 0x7f8eec10abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:50.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.351+0000 7f8ef1872640 1 --2- 172.21.15.181:0/1305532764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8eec103570 0x7f8eec105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:50.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.351+0000 7f8ef1872640 1 -- 172.21.15.181:0/1305532764 >> 172.21.15.181:0/1305532764 conn(0x7f8eec0fcea0 msgr2=0x7f8eec0ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:50.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.351+0000 7f8ef1872640 1 -- 172.21.15.181:0/1305532764 shutdown_connections 2024-05-03T20:36:50.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.351+0000 7f8ef1872640 1 -- 172.21.15.181:0/1305532764 wait complete. 2024-05-03T20:36:50.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.351+0000 7f8ef1872640 1 Processor -- start 2024-05-03T20:36:50.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.352+0000 7f8ef1872640 1 -- start start 2024-05-03T20:36:50.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.352+0000 7f8ef1872640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8eec103570 0x7f8eec198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:50.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.352+0000 7f8ef1872640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8eec105ea0 0x7f8eec199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:50.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.352+0000 7f8ef1872640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eec1087d0 0x7f8eec1a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:50.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.352+0000 7f8ef1872640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec10e540 con 0x7f8eec103570 2024-05-03T20:36:50.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.352+0000 7f8ef1872640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec10e3c0 con 0x7f8eec1087d0 2024-05-03T20:36:50.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.352+0000 7f8ef1872640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec10e6c0 con 0x7f8eec105ea0 2024-05-03T20:36:50.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.352+0000 7f8eeaffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8eec103570 0x7f8eec198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:50.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.352+0000 7f8eeaffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8eec103570 0x7f8eec198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36546/0 (socket says 172.21.15.181:36546) 2024-05-03T20:36:50.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.352+0000 7f8eeaffd640 1 -- 172.21.15.181:0/3149786716 learned_addr learned my addr 172.21.15.181:0/3149786716 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:50.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.352+0000 7f8eea7fc640 1 --2- 172.21.15.181:0/3149786716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8eec105ea0 0x7f8eec199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:50.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.352+0000 7f8eeb7fe640 1 --2- 172.21.15.181:0/3149786716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eec1087d0 0x7f8eec1a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:50.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.352+0000 7f8eeaffd640 1 -- 172.21.15.181:0/3149786716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8eec105ea0 msgr2=0x7f8eec199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:50.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.352+0000 7f8eeaffd640 1 --2- 172.21.15.181:0/3149786716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8eec105ea0 0x7f8eec199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:50.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.352+0000 7f8eeaffd640 1 -- 172.21.15.181:0/3149786716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eec1087d0 msgr2=0x7f8eec1a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:50.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.352+0000 7f8eeaffd640 1 --2- 172.21.15.181:0/3149786716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eec1087d0 0x7f8eec1a0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:50.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.352+0000 7f8eeaffd640 1 -- 172.21.15.181:0/3149786716 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eec1a07b0 con 0x7f8eec103570 2024-05-03T20:36:50.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.352+0000 7f8eea7fc640 1 --2- 172.21.15.181:0/3149786716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8eec105ea0 0x7f8eec199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:36:50.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.352+0000 7f8eeaffd640 1 --2- 172.21.15.181:0/3149786716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8eec103570 0x7f8eec198be0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8edc00cce0 tx=0x7f8edc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:50.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.353+0000 7f8ef0870640 1 -- 172.21.15.181:0/3149786716 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc00f650 con 0x7f8eec103570 2024-05-03T20:36:50.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.353+0000 7f8ef0870640 1 -- 172.21.15.181:0/3149786716 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8edc00fca0 con 0x7f8eec103570 2024-05-03T20:36:50.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.353+0000 7f8ef1872640 1 -- 172.21.15.181:0/3149786716 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eec1a0aa0 con 0x7f8eec103570 2024-05-03T20:36:50.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.353+0000 7f8ef0870640 1 -- 172.21.15.181:0/3149786716 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc018b40 con 0x7f8eec103570 2024-05-03T20:36:50.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.353+0000 7f8ef1872640 1 -- 172.21.15.181:0/3149786716 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eec1a0ff0 con 0x7f8eec103570 2024-05-03T20:36:50.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.354+0000 7f8ef1872640 1 -- 172.21.15.181:0/3149786716 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eec0fe990 con 0x7f8eec103570 2024-05-03T20:36:50.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.355+0000 7f8ef0870640 1 -- 172.21.15.181:0/3149786716 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f8edc00f7f0 con 0x7f8eec103570 2024-05-03T20:36:50.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.356+0000 7f8ef0870640 1 --2- 172.21.15.181:0/3149786716 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8ed40b2920 0x7f8ed40b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:50.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.356+0000 7f8eea7fc640 1 --2- 172.21.15.181:0/3149786716 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8ed40b2920 0x7f8ed40b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:50.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.356+0000 7f8ef0870640 1 -- 172.21.15.181:0/3149786716 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f8edc0e2900 con 0x7f8eec103570 2024-05-03T20:36:50.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.356+0000 7f8eea7fc640 1 --2- 172.21.15.181:0/3149786716 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8ed40b2920 0x7f8ed40b4de0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0006010 tx=0x7f8ee0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:50.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.361+0000 7f8ef0870640 1 -- 172.21.15.181:0/3149786716 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8edc014030 con 0x7f8eec103570 2024-05-03T20:36:50.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:50.584+0000 7f8ef1872640 1 -- 172.21.15.181:0/3149786716 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f8eec02db50 con 0x7f8eec103570 2024-05-03T20:36:51.434 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.436+0000 7f8ef0870640 1 -- 172.21.15.181:0/3149786716 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v28) ==== 118+0+0 (secure 0 0 0) 0x7f8edc0aa4c0 con 0x7f8eec103570 2024-05-03T20:36:51.437 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.439+0000 7f8ef1872640 1 -- 172.21.15.181:0/3149786716 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8ed40b2920 msgr2=0x7f8ed40b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:51.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.439+0000 7f8ef1872640 1 --2- 172.21.15.181:0/3149786716 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8ed40b2920 0x7f8ed40b4de0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0006010 tx=0x7f8ee0000bc0 comp rx=0 tx=0).stop 2024-05-03T20:36:51.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.439+0000 7f8ef1872640 1 -- 172.21.15.181:0/3149786716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8eec103570 msgr2=0x7f8eec198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:51.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.439+0000 7f8ef1872640 1 --2- 172.21.15.181:0/3149786716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8eec103570 0x7f8eec198be0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8edc00cce0 tx=0x7f8edc00a620 comp rx=0 tx=0).stop 2024-05-03T20:36:51.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.439+0000 7f8ef1872640 1 -- 172.21.15.181:0/3149786716 shutdown_connections 2024-05-03T20:36:51.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.439+0000 7f8ef1872640 1 --2- 172.21.15.181:0/3149786716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eec1087d0 0x7f8eec1a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:51.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.439+0000 7f8ef1872640 1 --2- 172.21.15.181:0/3149786716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8eec105ea0 0x7f8eec199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:51.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.439+0000 7f8ef1872640 1 --2- 172.21.15.181:0/3149786716 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f8ed40b2920 0x7f8ed40b4de0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:51.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.439+0000 7f8ef1872640 1 --2- 172.21.15.181:0/3149786716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8eec103570 0x7f8eec198be0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:51.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.439+0000 7f8ef1872640 1 -- 172.21.15.181:0/3149786716 >> 172.21.15.181:0/3149786716 conn(0x7f8eec0fcea0 msgr2=0x7f8eec103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:51.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.439+0000 7f8ef1872640 1 -- 172.21.15.181:0/3149786716 shutdown_connections 2024-05-03T20:36:51.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.439+0000 7f8ef1872640 1 -- 172.21.15.181:0/3149786716 wait complete. 2024-05-03T20:36:51.451 DEBUG:teuthology.orchestra.run.smithi181:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-05-03T20:36:51.559 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.558+0000 7f422d81d640 1 -- 172.21.15.181:0/2515483940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4228102b20 msgr2=0x7f4228102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:51.559 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.558+0000 7f422d81d640 1 --2- 172.21.15.181:0/2515483940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4228102b20 0x7f4228102fa0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f421c009900 tx=0x7f421c02a410 comp rx=0 tx=0).stop 2024-05-03T20:36:51.559 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.558+0000 7f422d81d640 1 -- 172.21.15.181:0/2515483940 shutdown_connections 2024-05-03T20:36:51.559 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.559+0000 7f422d81d640 1 --2- 172.21.15.181:0/2515483940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4228102b20 0x7f4228102fa0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:51.559 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.559+0000 7f422d81d640 1 --2- 172.21.15.181:0/2515483940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42281034e0 0x7f4228109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:51.559 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.559+0000 7f422d81d640 1 --2- 172.21.15.181:0/2515483940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4228101930 0x7f4228101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:51.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.559+0000 7f422d81d640 1 -- 172.21.15.181:0/2515483940 >> 172.21.15.181:0/2515483940 conn(0x7f42280fcec0 msgr2=0x7f42280ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:36:51.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.559+0000 7f422d81d640 1 -- 172.21.15.181:0/2515483940 shutdown_connections 2024-05-03T20:36:51.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.559+0000 7f422d81d640 1 -- 172.21.15.181:0/2515483940 wait complete. 2024-05-03T20:36:51.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.559+0000 7f422d81d640 1 Processor -- start 2024-05-03T20:36:51.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.559+0000 7f422d81d640 1 -- start start 2024-05-03T20:36:51.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.559+0000 7f422d81d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4228101930 0x7f42281058c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:51.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.559+0000 7f422d81d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4228102b20 0x7f4228103f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:51.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.559+0000 7f422d81d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f42281034e0 0x7f4228104440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:51.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.559+0000 7f422d81d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f422810d4c0 con 0x7f4228102b20 2024-05-03T20:36:51.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.559+0000 7f422d81d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f422810d340 con 0x7f42281034e0 2024-05-03T20:36:51.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.559+0000 7f422d81d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f422810d640 con 0x7f4228101930 2024-05-03T20:36:51.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.560+0000 7f42267fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4228102b20 0x7f4228103f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:51.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.560+0000 7f42267fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4228102b20 0x7f4228103f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36558/0 (socket says 172.21.15.181:36558) 2024-05-03T20:36:51.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.560+0000 7f42267fc640 1 -- 172.21.15.181:0/3015235437 learned_addr learned my addr 172.21.15.181:0/3015235437 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:36:51.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.560+0000 7f4226ffd640 1 --2- 172.21.15.181:0/3015235437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4228101930 0x7f42281058c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:51.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.560+0000 7f42267fc640 1 -- 172.21.15.181:0/3015235437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4228101930 msgr2=0x7f42281058c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:51.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.560+0000 7f42267fc640 1 --2- 172.21.15.181:0/3015235437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4228101930 0x7f42281058c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:51.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.560+0000 7f42267fc640 1 -- 172.21.15.181:0/3015235437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f42281034e0 msgr2=0x7f4228104440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:36:51.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.560+0000 7f42267fc640 1 --2- 172.21.15.181:0/3015235437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f42281034e0 0x7f4228104440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:36:51.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.560+0000 7f42267fc640 1 -- 172.21.15.181:0/3015235437 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4228104cd0 con 0x7f4228102b20 2024-05-03T20:36:51.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.560+0000 7f42267fc640 1 --2- 172.21.15.181:0/3015235437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4228102b20 0x7f4228103f00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f421c031500 tx=0x7f421c0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:51.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.560+0000 7f422c81b640 1 -- 172.21.15.181:0/3015235437 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c03f030 con 0x7f4228102b20 2024-05-03T20:36:51.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.560+0000 7f422c81b640 1 -- 172.21.15.181:0/3015235437 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f421c003800 con 0x7f4228102b20 2024-05-03T20:36:51.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.560+0000 7f422d81d640 1 -- 172.21.15.181:0/3015235437 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42281a2cf0 con 0x7f4228102b20 2024-05-03T20:36:51.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.560+0000 7f422c81b640 1 -- 172.21.15.181:0/3015235437 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c035070 con 0x7f4228102b20 2024-05-03T20:36:51.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.560+0000 7f422d81d640 1 -- 172.21.15.181:0/3015235437 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42281a3210 con 0x7f4228102b20 2024-05-03T20:36:51.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.561+0000 7f422d81d640 1 -- 172.21.15.181:0/3015235437 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42281a3440 con 0x7f4228102b20 2024-05-03T20:36:51.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.562+0000 7f422c81b640 1 -- 172.21.15.181:0/3015235437 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f421c0039b0 con 0x7f4228102b20 2024-05-03T20:36:51.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.563+0000 7f422c81b640 1 --2- 172.21.15.181:0/3015235437 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f42100b2920 0x7f42100b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:36:51.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.563+0000 7f4226ffd640 1 --2- 172.21.15.181:0/3015235437 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f42100b2920 0x7f42100b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:36:51.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.563+0000 7f422c81b640 1 -- 172.21.15.181:0/3015235437 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f421c103450 con 0x7f4228102b20 2024-05-03T20:36:51.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.564+0000 7f4226ffd640 1 --2- 172.21.15.181:0/3015235437 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f42100b2920 0x7f42100b4de0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4218005db0 tx=0x7f4218005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:36:51.565 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.567+0000 7f422c81b640 1 -- 172.21.15.181:0/3015235437 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f421c0cb010 con 0x7f4228102b20 2024-05-03T20:36:51.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:51 smithi181 ceph-mon[29801]: pgmap v381: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-03T20:36:51.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:51 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3149786716' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-03T20:36:51.776 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:36:51.777+0000 7f422d81d640 1 -- 172.21.15.181:0/3015235437 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f42280ff060 con 0x7f42100b2920 2024-05-03T20:36:51.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:51 smithi196 ceph-mon[32858]: pgmap v381: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-03T20:36:51.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:51 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3149786716' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-03T20:36:51.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:51 smithi204 ceph-mon[33595]: pgmap v381: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-03T20:36:51.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:51 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3149786716' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-03T20:36:52.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:52 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3149786716' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-03T20:36:52.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:52 smithi181 ceph-mon[29801]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-03T20:36:52.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:52 smithi181 ceph-mon[29801]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:36:52.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:52 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3149786716' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-03T20:36:52.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:52 smithi196 ceph-mon[32858]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-03T20:36:52.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:52 smithi196 ceph-mon[32858]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:36:52.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:52 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3149786716' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-03T20:36:52.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:52 smithi204 ceph-mon[33595]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-03T20:36:52.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:52 smithi204 ceph-mon[33595]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:36:53.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:53 smithi181 ceph-mon[29801]: pgmap v382: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-03T20:36:53.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:53 smithi196 ceph-mon[32858]: pgmap v382: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-03T20:36:53.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:53 smithi204 ceph-mon[33595]: pgmap v382: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-03T20:36:55.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:55 smithi181 ceph-mon[29801]: pgmap v383: 97 pgs: 97 active+clean; 637 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-03T20:36:55.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:55 smithi196 ceph-mon[32858]: pgmap v383: 97 pgs: 97 active+clean; 637 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-03T20:36:55.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:55 smithi204 ceph-mon[33595]: pgmap v383: 97 pgs: 97 active+clean; 637 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-03T20:36:57.489 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:57 smithi181 ceph-mon[29801]: pgmap v384: 97 pgs: 97 active+clean; 637 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-03T20:36:57.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:57 smithi196 ceph-mon[32858]: pgmap v384: 97 pgs: 97 active+clean; 637 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-03T20:36:57.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:57 smithi204 ceph-mon[33595]: pgmap v384: 97 pgs: 97 active+clean; 637 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-03T20:36:59.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:36:59 smithi181 ceph-mon[29801]: pgmap v385: 97 pgs: 97 active+clean; 637 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-03T20:36:59.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:36:59 smithi196 ceph-mon[32858]: pgmap v385: 97 pgs: 97 active+clean; 637 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-03T20:36:59.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:36:59 smithi204 ceph-mon[33595]: pgmap v385: 97 pgs: 97 active+clean; 637 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-03T20:37:01.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:01 smithi181 ceph-mon[29801]: pgmap v386: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 853 B/s wr, 11 op/s 2024-05-03T20:37:01.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:01 smithi196 ceph-mon[32858]: pgmap v386: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 853 B/s wr, 11 op/s 2024-05-03T20:37:01.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:01 smithi204 ceph-mon[33595]: pgmap v386: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 853 B/s wr, 11 op/s 2024-05-03T20:37:03.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:03 smithi181 ceph-mon[29801]: pgmap v387: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 853 B/s wr, 9 op/s 2024-05-03T20:37:03.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:03 smithi196 ceph-mon[32858]: pgmap v387: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 853 B/s wr, 9 op/s 2024-05-03T20:37:03.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:03 smithi204 ceph-mon[33595]: pgmap v387: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 853 B/s wr, 9 op/s 2024-05-03T20:37:05.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:05 smithi181 ceph-mon[29801]: pgmap v388: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-03T20:37:05.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:05 smithi196 ceph-mon[32858]: pgmap v388: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-03T20:37:05.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:05 smithi204 ceph-mon[33595]: pgmap v388: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-03T20:37:06.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:06 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:37:06.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:06 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:37:06.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:06 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:37:06.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:06 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:37:06.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:06 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:37:06.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:06 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:37:06.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:06 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:37:06.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:06 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:37:06.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:06 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:37:06.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:06 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:37:06.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:06 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:37:06.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:06 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:37:06.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:06 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:37:06.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:06 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:37:06.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:06 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:37:07.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:07 smithi196 ceph-mon[32858]: pgmap v389: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-03T20:37:07.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:37:07.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:37:07.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:37:07.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:07 smithi204 ceph-mon[33595]: pgmap v389: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-03T20:37:07.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:37:07.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:37:07.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:37:07.931 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:07 smithi181 ceph-mon[29801]: pgmap v389: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-03T20:37:07.931 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:37:07.931 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:37:07.931 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:37:09.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:09 smithi196 ceph-mon[32858]: pgmap v390: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-03T20:37:09.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:09 smithi204 ceph-mon[33595]: pgmap v390: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-03T20:37:09.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:09 smithi181 ceph-mon[29801]: pgmap v390: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-03T20:37:11.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:11 smithi196 ceph-mon[32858]: pgmap v391: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-03T20:37:11.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:11 smithi204 ceph-mon[33595]: pgmap v391: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-03T20:37:11.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:11 smithi181 ceph-mon[29801]: pgmap v391: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-03T20:37:13.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:13 smithi196 ceph-mon[32858]: pgmap v392: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:37:13.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:13 smithi204 ceph-mon[33595]: pgmap v392: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:37:13.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:13 smithi181 ceph-mon[29801]: pgmap v392: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:37:15.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:15 smithi196 ceph-mon[32858]: pgmap v393: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:37:15.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:15 smithi204 ceph-mon[33595]: pgmap v393: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:37:15.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:15 smithi181 ceph-mon[29801]: pgmap v393: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:37:17.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:17 smithi196 ceph-mon[32858]: pgmap v394: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:37:17.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:17 smithi204 ceph-mon[33595]: pgmap v394: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:37:17.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:17 smithi181 ceph-mon[29801]: pgmap v394: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:37:19.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:19 smithi196 ceph-mon[32858]: pgmap v395: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:37:19.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:19 smithi204 ceph-mon[33595]: pgmap v395: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:37:19.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:19 smithi181 ceph-mon[29801]: pgmap v395: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:37:21.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:21 smithi196 ceph-mon[32858]: pgmap v396: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:37:21.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:21 smithi204 ceph-mon[33595]: pgmap v396: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:37:21.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:21 smithi181 ceph-mon[29801]: pgmap v396: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:37:23.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:23 smithi196 ceph-mon[32858]: pgmap v397: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:37:23.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:23 smithi204 ceph-mon[33595]: pgmap v397: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:37:23.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:23 smithi181 ceph-mon[29801]: pgmap v397: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:37:25.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:25 smithi196 ceph-mon[32858]: pgmap v398: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:37:25.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:25 smithi204 ceph-mon[33595]: pgmap v398: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:37:25.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:25 smithi181 ceph-mon[29801]: pgmap v398: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:37:27.764 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:27 smithi181 ceph-mon[29801]: pgmap v399: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:37:27.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:27 smithi196 ceph-mon[32858]: pgmap v399: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:37:27.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:27 smithi204 ceph-mon[33595]: pgmap v399: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:37:29.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:29 smithi196 ceph-mon[32858]: pgmap v400: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:37:29.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:29 smithi204 ceph-mon[33595]: pgmap v400: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:37:29.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:29 smithi181 ceph-mon[29801]: pgmap v400: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:37:31.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:31 smithi196 ceph-mon[32858]: pgmap v401: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:37:31.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:31 smithi204 ceph-mon[33595]: pgmap v401: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:37:31.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:31 smithi181 ceph-mon[29801]: pgmap v401: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:37:33.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:33 smithi196 ceph-mon[32858]: pgmap v402: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:37:33.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:33 smithi204 ceph-mon[33595]: pgmap v402: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:37:33.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:33 smithi181 ceph-mon[29801]: pgmap v402: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:37:35.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:35 smithi196 ceph-mon[32858]: pgmap v403: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:37:35.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:35 smithi204 ceph-mon[33595]: pgmap v403: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:37:35.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:35 smithi181 ceph-mon[29801]: pgmap v403: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:37:37.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:37 smithi196 ceph-mon[32858]: pgmap v404: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:37:37.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:37 smithi204 ceph-mon[33595]: pgmap v404: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:37:37.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:37 smithi181 ceph-mon[29801]: pgmap v404: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:37:39.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:39 smithi196 ceph-mon[32858]: pgmap v405: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:37:39.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:39 smithi204 ceph-mon[33595]: pgmap v405: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:37:39.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:39 smithi181 ceph-mon[29801]: pgmap v405: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:37:41.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:41 smithi196 ceph-mon[32858]: pgmap v406: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:37:41.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:41 smithi204 ceph-mon[33595]: pgmap v406: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:37:41.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:41 smithi181 ceph-mon[29801]: pgmap v406: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:37:43.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:43 smithi196 ceph-mon[32858]: pgmap v407: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:37:43.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:43 smithi204 ceph-mon[33595]: pgmap v407: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:37:43.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:43 smithi181 ceph-mon[29801]: pgmap v407: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:37:45.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:45 smithi196 ceph-mon[32858]: pgmap v408: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:37:45.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:45 smithi204 ceph-mon[33595]: pgmap v408: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:37:45.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:45 smithi181 ceph-mon[29801]: pgmap v408: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:37:47.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:47 smithi196 ceph-mon[32858]: pgmap v409: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:37:47.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:47 smithi204 ceph-mon[33595]: pgmap v409: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:37:47.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:47 smithi181 ceph-mon[29801]: pgmap v409: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:37:49.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:49 smithi196 ceph-mon[32858]: pgmap v410: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:37:49.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:49 smithi204 ceph-mon[33595]: pgmap v410: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:37:49.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:49 smithi181 ceph-mon[29801]: pgmap v410: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:37:51.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:51 smithi196 ceph-mon[32858]: pgmap v411: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:37:51.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:51 smithi204 ceph-mon[33595]: pgmap v411: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:37:51.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:51 smithi181 ceph-mon[29801]: pgmap v411: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:37:53.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:53 smithi196 ceph-mon[32858]: pgmap v412: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:37:53.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:53 smithi204 ceph-mon[33595]: pgmap v412: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:37:53.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:53 smithi181 ceph-mon[29801]: pgmap v412: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:37:55.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:55 smithi196 ceph-mon[32858]: pgmap v413: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:37:55.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:55 smithi204 ceph-mon[33595]: pgmap v413: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:37:55.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:55 smithi181 ceph-mon[29801]: pgmap v413: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:37:57.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:57 smithi196 ceph-mon[32858]: pgmap v414: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-03T20:37:57.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:57 smithi204 ceph-mon[33595]: pgmap v414: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-03T20:37:57.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:57 smithi181 ceph-mon[29801]: pgmap v414: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-03T20:37:59.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:37:59 smithi196 ceph-mon[32858]: pgmap v415: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-03T20:37:59.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:37:59 smithi204 ceph-mon[33595]: pgmap v415: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-03T20:37:59.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:37:59 smithi181 ceph-mon[29801]: pgmap v415: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-03T20:38:01.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:01 smithi196 ceph-mon[32858]: pgmap v416: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-03T20:38:01.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:01 smithi204 ceph-mon[33595]: pgmap v416: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-03T20:38:01.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:01 smithi181 ceph-mon[29801]: pgmap v416: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-03T20:38:03.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:03 smithi196 ceph-mon[32858]: pgmap v417: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-03T20:38:03.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:03 smithi204 ceph-mon[33595]: pgmap v417: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-03T20:38:03.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:03 smithi181 ceph-mon[29801]: pgmap v417: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-03T20:38:05.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:05 smithi196 ceph-mon[32858]: pgmap v418: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2024-05-03T20:38:05.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:05 smithi204 ceph-mon[33595]: pgmap v418: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2024-05-03T20:38:05.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:05 smithi181 ceph-mon[29801]: pgmap v418: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2024-05-03T20:38:07.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:07 smithi196 ceph-mon[32858]: pgmap v419: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-03T20:38:07.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:38:07.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:38:07.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:38:07.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:07 smithi204 ceph-mon[33595]: pgmap v419: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-03T20:38:07.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:38:07.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:38:07.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:38:07.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:07 smithi181 ceph-mon[29801]: pgmap v419: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-03T20:38:07.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:38:07.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:38:07.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:38:08.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:08 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:38:08.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:08 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:38:08.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:08 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:38:08.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:08 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:38:08.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:08 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:38:08.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:08 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:38:08.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:08 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:38:08.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:08 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:38:08.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:08 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:38:09.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:09 smithi196 ceph-mon[32858]: pgmap v420: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:38:09.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:09 smithi204 ceph-mon[33595]: pgmap v420: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:38:09.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:09 smithi181 ceph-mon[29801]: pgmap v420: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:38:11.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:11 smithi196 ceph-mon[32858]: pgmap v421: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:38:11.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:11 smithi204 ceph-mon[33595]: pgmap v421: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:38:11.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:11 smithi181 ceph-mon[29801]: pgmap v421: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:38:13.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:13 smithi196 ceph-mon[32858]: pgmap v422: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:38:13.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:13 smithi204 ceph-mon[33595]: pgmap v422: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:38:13.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:13 smithi181 ceph-mon[29801]: pgmap v422: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:38:15.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:15 smithi196 ceph-mon[32858]: pgmap v423: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:38:15.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:15 smithi204 ceph-mon[33595]: pgmap v423: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:38:15.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:15 smithi181 ceph-mon[29801]: pgmap v423: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:38:17.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:17 smithi196 ceph-mon[32858]: pgmap v424: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:38:17.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:17 smithi204 ceph-mon[33595]: pgmap v424: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:38:17.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:17 smithi181 ceph-mon[29801]: pgmap v424: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:38:19.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:19 smithi196 ceph-mon[32858]: pgmap v425: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:38:19.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:19 smithi204 ceph-mon[33595]: pgmap v425: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:38:19.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:19 smithi181 ceph-mon[29801]: pgmap v425: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:38:21.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:21 smithi196 ceph-mon[32858]: pgmap v426: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:38:21.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:21 smithi204 ceph-mon[33595]: pgmap v426: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:38:21.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:21 smithi181 ceph-mon[29801]: pgmap v426: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:38:23.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:23 smithi196 ceph-mon[32858]: pgmap v427: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:38:23.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:23 smithi204 ceph-mon[33595]: pgmap v427: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:38:23.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:23 smithi181 ceph-mon[29801]: pgmap v427: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:38:25.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:25 smithi196 ceph-mon[32858]: pgmap v428: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:38:25.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:25 smithi204 ceph-mon[33595]: pgmap v428: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:38:25.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:25 smithi181 ceph-mon[29801]: pgmap v428: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:38:27.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:27 smithi196 ceph-mon[32858]: pgmap v429: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:38:27.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:27 smithi204 ceph-mon[33595]: pgmap v429: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:38:27.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:27 smithi181 ceph-mon[29801]: pgmap v429: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:38:29.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:29 smithi204 ceph-mon[33595]: pgmap v430: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:38:29.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:29 smithi181 ceph-mon[29801]: pgmap v430: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:38:30.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:29 smithi196 ceph-mon[32858]: pgmap v430: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:38:31.865 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:31 smithi204 ceph-mon[33595]: pgmap v431: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:38:31.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:31 smithi181 ceph-mon[29801]: pgmap v431: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:38:32.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:31 smithi196 ceph-mon[32858]: pgmap v431: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:38:33.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:33 smithi204 ceph-mon[33595]: pgmap v432: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:38:33.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:33 smithi181 ceph-mon[29801]: pgmap v432: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:38:34.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:33 smithi196 ceph-mon[32858]: pgmap v432: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:38:35.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:35 smithi181 ceph-mon[29801]: pgmap v433: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:38:36.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:35 smithi196 ceph-mon[32858]: pgmap v433: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:38:36.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:35 smithi204 ceph-mon[33595]: pgmap v433: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:38:37.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:37 smithi181 ceph-mon[29801]: pgmap v434: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T20:38:38.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:37 smithi196 ceph-mon[32858]: pgmap v434: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T20:38:38.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:37 smithi204 ceph-mon[33595]: pgmap v434: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T20:38:39.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:39 smithi181 ceph-mon[29801]: pgmap v435: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T20:38:40.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:39 smithi196 ceph-mon[32858]: pgmap v435: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T20:38:40.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:39 smithi204 ceph-mon[33595]: pgmap v435: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T20:38:41.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:41 smithi181 ceph-mon[29801]: pgmap v436: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-03T20:38:42.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:41 smithi196 ceph-mon[32858]: pgmap v436: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-03T20:38:42.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:41 smithi204 ceph-mon[33595]: pgmap v436: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-03T20:38:43.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:43 smithi204 ceph-mon[33595]: pgmap v437: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-03T20:38:43.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:43 smithi181 ceph-mon[29801]: pgmap v437: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-03T20:38:44.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:43 smithi196 ceph-mon[32858]: pgmap v437: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-03T20:38:45.908 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:45 smithi196 ceph-mon[32858]: pgmap v438: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-05-03T20:38:45.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:45 smithi181 ceph-mon[29801]: pgmap v438: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-05-03T20:38:46.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:45 smithi204 ceph-mon[33595]: pgmap v438: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-05-03T20:38:47.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:47 smithi181 ceph-mon[29801]: pgmap v439: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T20:38:48.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:47 smithi196 ceph-mon[32858]: pgmap v439: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T20:38:48.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:47 smithi204 ceph-mon[33595]: pgmap v439: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T20:38:49.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:49 smithi181 ceph-mon[29801]: pgmap v440: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:38:50.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:49 smithi196 ceph-mon[32858]: pgmap v440: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:38:50.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:49 smithi204 ceph-mon[33595]: pgmap v440: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:38:51.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:51 smithi181 ceph-mon[29801]: pgmap v441: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:38:52.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:51 smithi196 ceph-mon[32858]: pgmap v441: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:38:52.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:51 smithi204 ceph-mon[33595]: pgmap v441: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:38:53.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:53 smithi181 ceph-mon[29801]: pgmap v442: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:38:54.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:53 smithi196 ceph-mon[32858]: pgmap v442: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:38:54.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:53 smithi204 ceph-mon[33595]: pgmap v442: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:38:55.966 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:55 smithi181 ceph-mon[29801]: pgmap v443: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:38:56.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:55 smithi196 ceph-mon[32858]: pgmap v443: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:38:56.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:55 smithi204 ceph-mon[33595]: pgmap v443: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:38:57.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:57 smithi181 ceph-mon[29801]: pgmap v444: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:38:58.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:57 smithi196 ceph-mon[32858]: pgmap v444: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:38:58.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:57 smithi204 ceph-mon[33595]: pgmap v444: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:38:59.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:38:59 smithi181 ceph-mon[29801]: pgmap v445: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:00.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:38:59 smithi196 ceph-mon[32858]: pgmap v445: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:00.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:38:59 smithi204 ceph-mon[33595]: pgmap v445: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:01.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:01 smithi181 ceph-mon[29801]: pgmap v446: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:39:02.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:01 smithi196 ceph-mon[32858]: pgmap v446: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:39:02.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:01 smithi204 ceph-mon[33595]: pgmap v446: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:39:03.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:03 smithi181 ceph-mon[29801]: pgmap v447: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:39:04.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:03 smithi196 ceph-mon[32858]: pgmap v447: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:39:04.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:03 smithi204 ceph-mon[33595]: pgmap v447: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:39:05.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:05 smithi181 ceph-mon[29801]: pgmap v448: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:39:06.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:05 smithi196 ceph-mon[32858]: pgmap v448: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:39:06.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:05 smithi204 ceph-mon[33595]: pgmap v448: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:39:07.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:07 smithi181 ceph-mon[29801]: pgmap v449: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:08.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:07 smithi196 ceph-mon[32858]: pgmap v449: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:08.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:07 smithi204 ceph-mon[33595]: pgmap v449: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:08.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:08 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:39:09.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:08 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:39:09.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:08 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:39:09.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:09 smithi181 ceph-mon[29801]: pgmap v450: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:09.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:09 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:39:09.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:09 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:39:09.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:09 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:39:10.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:09 smithi196 ceph-mon[32858]: pgmap v450: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:10.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:09 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:39:10.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:09 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:39:10.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:09 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:39:10.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:09 smithi204 ceph-mon[33595]: pgmap v450: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:10.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:09 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:39:10.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:09 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:39:10.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:09 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:39:11.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:11 smithi181 ceph-mon[29801]: pgmap v451: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:39:12.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:11 smithi196 ceph-mon[32858]: pgmap v451: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:39:12.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:11 smithi204 ceph-mon[33595]: pgmap v451: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:39:13.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:13 smithi181 ceph-mon[29801]: pgmap v452: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:39:14.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:13 smithi196 ceph-mon[32858]: pgmap v452: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:39:14.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:13 smithi204 ceph-mon[33595]: pgmap v452: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:39:15.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:15 smithi181 ceph-mon[29801]: pgmap v453: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:39:16.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:15 smithi196 ceph-mon[32858]: pgmap v453: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:39:16.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:15 smithi204 ceph-mon[33595]: pgmap v453: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:39:17.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:17 smithi181 ceph-mon[29801]: pgmap v454: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:18.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:17 smithi196 ceph-mon[32858]: pgmap v454: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:18.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:17 smithi204 ceph-mon[33595]: pgmap v454: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:19.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:19 smithi181 ceph-mon[29801]: pgmap v455: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:20.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:19 smithi196 ceph-mon[32858]: pgmap v455: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:20.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:19 smithi204 ceph-mon[33595]: pgmap v455: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:21.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:21 smithi181 ceph-mon[29801]: pgmap v456: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:39:22.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:21 smithi196 ceph-mon[32858]: pgmap v456: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:39:22.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:21 smithi204 ceph-mon[33595]: pgmap v456: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:39:23.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:23 smithi181 ceph-mon[29801]: pgmap v457: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:39:24.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:23 smithi196 ceph-mon[32858]: pgmap v457: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:39:24.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:23 smithi204 ceph-mon[33595]: pgmap v457: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:39:25.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:25 smithi181 ceph-mon[29801]: pgmap v458: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:39:26.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:25 smithi196 ceph-mon[32858]: pgmap v458: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:39:26.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:25 smithi204 ceph-mon[33595]: pgmap v458: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:39:27.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:27 smithi181 ceph-mon[29801]: pgmap v459: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:28.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:27 smithi196 ceph-mon[32858]: pgmap v459: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:28.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:27 smithi204 ceph-mon[33595]: pgmap v459: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:29.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:29 smithi181 ceph-mon[29801]: pgmap v460: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:30.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:29 smithi196 ceph-mon[32858]: pgmap v460: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:30.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:29 smithi204 ceph-mon[33595]: pgmap v460: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:31.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:31 smithi181 ceph-mon[29801]: pgmap v461: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-03T20:39:32.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:31 smithi196 ceph-mon[32858]: pgmap v461: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-03T20:39:32.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:31 smithi204 ceph-mon[33595]: pgmap v461: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-03T20:39:33.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:33 smithi181 ceph-mon[29801]: pgmap v462: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-03T20:39:34.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:33 smithi196 ceph-mon[32858]: pgmap v462: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-03T20:39:34.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:33 smithi204 ceph-mon[33595]: pgmap v462: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-03T20:39:36.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:35 smithi196 ceph-mon[32858]: pgmap v463: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-05-03T20:39:36.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:35 smithi204 ceph-mon[33595]: pgmap v463: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-05-03T20:39:36.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:35 smithi181 ceph-mon[29801]: pgmap v463: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-05-03T20:39:38.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:37 smithi196 ceph-mon[32858]: pgmap v464: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T20:39:38.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:37 smithi204 ceph-mon[33595]: pgmap v464: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T20:39:38.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:37 smithi181 ceph-mon[29801]: pgmap v464: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T20:39:40.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:39 smithi196 ceph-mon[32858]: pgmap v465: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T20:39:40.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:39 smithi204 ceph-mon[33595]: pgmap v465: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T20:39:40.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:39 smithi181 ceph-mon[29801]: pgmap v465: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T20:39:42.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:41 smithi196 ceph-mon[32858]: pgmap v466: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-03T20:39:42.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:41 smithi204 ceph-mon[33595]: pgmap v466: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-03T20:39:42.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:41 smithi181 ceph-mon[29801]: pgmap v466: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-03T20:39:44.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:43 smithi196 ceph-mon[32858]: pgmap v467: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:39:44.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:43 smithi204 ceph-mon[33595]: pgmap v467: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:39:44.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:43 smithi181 ceph-mon[29801]: pgmap v467: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:39:46.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:45 smithi196 ceph-mon[32858]: pgmap v468: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:39:46.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:45 smithi204 ceph-mon[33595]: pgmap v468: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:39:46.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:45 smithi181 ceph-mon[29801]: pgmap v468: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:39:48.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:47 smithi196 ceph-mon[32858]: pgmap v469: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:48.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:47 smithi204 ceph-mon[33595]: pgmap v469: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:48.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:47 smithi181 ceph-mon[29801]: pgmap v469: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:50.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:49 smithi196 ceph-mon[32858]: pgmap v470: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:50.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:49 smithi204 ceph-mon[33595]: pgmap v470: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:50.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:49 smithi181 ceph-mon[29801]: pgmap v470: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:51.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:39:51.583+0000 7f420f7fe640 1 -- 172.21.15.181:0/3015235437 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f42080034c0 con 0x7f4228102b20 2024-05-03T20:39:51.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:39:51.583+0000 7f422c81b640 1 -- 172.21.15.181:0/3015235437 <== mon.0 v2:172.21.15.181:3300/0 7 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f42080034c0 con 0x7f4228102b20 2024-05-03T20:39:52.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:51 smithi196 ceph-mon[32858]: pgmap v471: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:39:52.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:51 smithi204 ceph-mon[33595]: pgmap v471: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:39:52.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:51 smithi181 ceph-mon[29801]: pgmap v471: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:39:54.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:53 smithi196 ceph-mon[32858]: pgmap v472: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:39:54.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:53 smithi204 ceph-mon[33595]: pgmap v472: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:39:54.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:53 smithi181 ceph-mon[29801]: pgmap v472: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:39:56.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:55 smithi196 ceph-mon[32858]: pgmap v473: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:39:56.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:55 smithi204 ceph-mon[33595]: pgmap v473: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:39:56.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:55 smithi181 ceph-mon[29801]: pgmap v473: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:39:58.014 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:57 smithi181 ceph-mon[29801]: pgmap v474: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:58.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:57 smithi196 ceph-mon[32858]: pgmap v474: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:39:58.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:57 smithi204 ceph-mon[33595]: pgmap v474: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:00.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:39:59 smithi196 ceph-mon[32858]: pgmap v475: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:00.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:39:59 smithi204 ceph-mon[33595]: pgmap v475: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:00.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:39:59 smithi181 ceph-mon[29801]: pgmap v475: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:01.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:00 smithi196 ceph-mon[32858]: overall HEALTH_OK 2024-05-03T20:40:01.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:00 smithi204 ceph-mon[33595]: overall HEALTH_OK 2024-05-03T20:40:01.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:00 smithi181 ceph-mon[29801]: overall HEALTH_OK 2024-05-03T20:40:02.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:01 smithi196 ceph-mon[32858]: pgmap v476: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:40:02.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:01 smithi204 ceph-mon[33595]: pgmap v476: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:40:02.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:01 smithi181 ceph-mon[29801]: pgmap v476: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:40:04.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:03 smithi196 ceph-mon[32858]: pgmap v477: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:40:04.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:03 smithi204 ceph-mon[33595]: pgmap v477: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:40:04.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:03 smithi181 ceph-mon[29801]: pgmap v477: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:40:06.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:05 smithi196 ceph-mon[32858]: pgmap v478: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:40:06.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:05 smithi204 ceph-mon[33595]: pgmap v478: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:40:06.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:05 smithi181 ceph-mon[29801]: pgmap v478: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:40:08.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:07 smithi196 ceph-mon[32858]: pgmap v479: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:08.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:07 smithi204 ceph-mon[33595]: pgmap v479: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:08.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:07 smithi181 ceph-mon[29801]: pgmap v479: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:10.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:09 smithi196 ceph-mon[32858]: pgmap v480: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:10.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:09 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:40:10.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:09 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:40:10.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:09 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:40:10.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:09 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:40:10.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:09 smithi204 ceph-mon[33595]: pgmap v480: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:10.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:09 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:40:10.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:09 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:40:10.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:09 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:40:10.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:09 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:40:10.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:09 smithi181 ceph-mon[29801]: pgmap v480: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:10.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:09 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:40:10.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:09 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:40:10.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:09 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:40:10.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:09 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:40:12.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:11 smithi196 ceph-mon[32858]: pgmap v481: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:40:12.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:11 smithi204 ceph-mon[33595]: pgmap v481: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:40:12.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:11 smithi181 ceph-mon[29801]: pgmap v481: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:40:14.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:13 smithi196 ceph-mon[32858]: pgmap v482: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:40:14.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:13 smithi204 ceph-mon[33595]: pgmap v482: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:40:14.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:13 smithi181 ceph-mon[29801]: pgmap v482: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:40:16.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:15 smithi196 ceph-mon[32858]: pgmap v483: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:40:16.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:15 smithi204 ceph-mon[33595]: pgmap v483: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:40:16.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:15 smithi181 ceph-mon[29801]: pgmap v483: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:40:18.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:17 smithi196 ceph-mon[32858]: pgmap v484: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:18.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:17 smithi204 ceph-mon[33595]: pgmap v484: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:18.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:17 smithi181 ceph-mon[29801]: pgmap v484: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:20.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:19 smithi196 ceph-mon[32858]: pgmap v485: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:20.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:19 smithi204 ceph-mon[33595]: pgmap v485: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:20.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:19 smithi181 ceph-mon[29801]: pgmap v485: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:22.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:21 smithi196 ceph-mon[32858]: pgmap v486: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:40:22.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:21 smithi204 ceph-mon[33595]: pgmap v486: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:40:22.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:21 smithi181 ceph-mon[29801]: pgmap v486: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:40:24.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:23 smithi196 ceph-mon[32858]: pgmap v487: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:40:24.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:23 smithi204 ceph-mon[33595]: pgmap v487: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:40:24.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:23 smithi181 ceph-mon[29801]: pgmap v487: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:40:26.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:25 smithi196 ceph-mon[32858]: pgmap v488: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:40:26.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:25 smithi204 ceph-mon[33595]: pgmap v488: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:40:26.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:25 smithi181 ceph-mon[29801]: pgmap v488: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:40:28.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:27 smithi196 ceph-mon[32858]: pgmap v489: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:28.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:27 smithi204 ceph-mon[33595]: pgmap v489: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:28.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:27 smithi181 ceph-mon[29801]: pgmap v489: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:30.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:29 smithi196 ceph-mon[32858]: pgmap v490: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:30.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:29 smithi204 ceph-mon[33595]: pgmap v490: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:30.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:29 smithi181 ceph-mon[29801]: pgmap v490: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:32.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:31 smithi196 ceph-mon[32858]: pgmap v491: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:40:32.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:31 smithi204 ceph-mon[33595]: pgmap v491: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:40:32.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:31 smithi181 ceph-mon[29801]: pgmap v491: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:40:34.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:33 smithi196 ceph-mon[32858]: pgmap v492: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:40:34.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:33 smithi204 ceph-mon[33595]: pgmap v492: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:40:34.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:33 smithi181 ceph-mon[29801]: pgmap v492: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:40:36.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:35 smithi196 ceph-mon[32858]: pgmap v493: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:40:36.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:35 smithi204 ceph-mon[33595]: pgmap v493: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:40:36.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:35 smithi181 ceph-mon[29801]: pgmap v493: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:40:38.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:37 smithi196 ceph-mon[32858]: pgmap v494: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:38.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:37 smithi204 ceph-mon[33595]: pgmap v494: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:38.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:37 smithi181 ceph-mon[29801]: pgmap v494: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:40.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:39 smithi196 ceph-mon[32858]: pgmap v495: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:40.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:39 smithi204 ceph-mon[33595]: pgmap v495: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:40.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:39 smithi181 ceph-mon[29801]: pgmap v495: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:42.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:41 smithi196 ceph-mon[32858]: pgmap v496: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:40:42.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:41 smithi204 ceph-mon[33595]: pgmap v496: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:40:42.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:41 smithi181 ceph-mon[29801]: pgmap v496: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:40:44.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:43 smithi196 ceph-mon[32858]: pgmap v497: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:40:44.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:43 smithi204 ceph-mon[33595]: pgmap v497: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:40:44.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:43 smithi181 ceph-mon[29801]: pgmap v497: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:40:46.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:45 smithi196 ceph-mon[32858]: pgmap v498: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:40:46.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:45 smithi204 ceph-mon[33595]: pgmap v498: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:40:46.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:45 smithi181 ceph-mon[29801]: pgmap v498: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:40:48.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:47 smithi196 ceph-mon[32858]: pgmap v499: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:48.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:47 smithi204 ceph-mon[33595]: pgmap v499: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:48.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:47 smithi181 ceph-mon[29801]: pgmap v499: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:50.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:49 smithi196 ceph-mon[32858]: pgmap v500: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:50.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:49 smithi204 ceph-mon[33595]: pgmap v500: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:50.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:49 smithi181 ceph-mon[29801]: pgmap v500: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:52.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:51 smithi196 ceph-mon[32858]: pgmap v501: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:40:52.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:51 smithi204 ceph-mon[33595]: pgmap v501: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:40:52.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:51 smithi181 ceph-mon[29801]: pgmap v501: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:40:54.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:53 smithi196 ceph-mon[32858]: pgmap v502: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:40:54.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:53 smithi204 ceph-mon[33595]: pgmap v502: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:40:54.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:53 smithi181 ceph-mon[29801]: pgmap v502: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:40:56.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:55 smithi196 ceph-mon[32858]: pgmap v503: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:40:56.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:55 smithi204 ceph-mon[33595]: pgmap v503: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:40:56.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:55 smithi181 ceph-mon[29801]: pgmap v503: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:40:58.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:57 smithi196 ceph-mon[32858]: pgmap v504: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:58.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:57 smithi204 ceph-mon[33595]: pgmap v504: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:40:58.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:57 smithi181 ceph-mon[29801]: pgmap v504: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:00.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:40:59 smithi196 ceph-mon[32858]: pgmap v505: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:00.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:40:59 smithi204 ceph-mon[33595]: pgmap v505: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:00.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:40:59 smithi181 ceph-mon[29801]: pgmap v505: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:02.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:01 smithi196 ceph-mon[32858]: pgmap v506: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:41:02.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:01 smithi204 ceph-mon[33595]: pgmap v506: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:41:02.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:01 smithi181 ceph-mon[29801]: pgmap v506: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:41:04.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:03 smithi204 ceph-mon[33595]: pgmap v507: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:41:04.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:03 smithi181 ceph-mon[29801]: pgmap v507: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:41:04.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:03 smithi196 ceph-mon[32858]: pgmap v507: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:41:06.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:05 smithi204 ceph-mon[33595]: pgmap v508: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:41:06.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:05 smithi181 ceph-mon[29801]: pgmap v508: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:41:06.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:05 smithi196 ceph-mon[32858]: pgmap v508: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:41:08.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:07 smithi204 ceph-mon[33595]: pgmap v509: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:08.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:07 smithi181 ceph-mon[29801]: pgmap v509: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:08.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:07 smithi196 ceph-mon[32858]: pgmap v509: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:10.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:09 smithi181 ceph-mon[29801]: pgmap v510: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:10.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:09 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:41:10.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:09 smithi196 ceph-mon[32858]: pgmap v510: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:10.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:09 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:41:10.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:09 smithi204 ceph-mon[33595]: pgmap v510: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:10.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:09 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:41:11.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:10 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:41:11.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:10 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:41:11.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:10 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:41:11.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:10 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:41:11.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:10 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:41:11.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:10 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:41:11.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:10 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:41:11.366 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:10 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:41:11.366 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:10 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:41:12.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:11 smithi181 ceph-mon[29801]: pgmap v511: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:41:12.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:11 smithi196 ceph-mon[32858]: pgmap v511: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:41:12.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:11 smithi204 ceph-mon[33595]: pgmap v511: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:41:14.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:13 smithi204 ceph-mon[33595]: pgmap v512: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:41:14.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:13 smithi181 ceph-mon[29801]: pgmap v512: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:41:14.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:13 smithi196 ceph-mon[32858]: pgmap v512: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:41:16.158 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:15 smithi196 ceph-mon[32858]: pgmap v513: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-03T20:41:16.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:15 smithi181 ceph-mon[29801]: pgmap v513: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-03T20:41:16.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:15 smithi204 ceph-mon[33595]: pgmap v513: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-03T20:41:18.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:17 smithi181 ceph-mon[29801]: pgmap v514: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-03T20:41:18.351 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:17 smithi196 ceph-mon[32858]: pgmap v514: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-03T20:41:18.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:17 smithi204 ceph-mon[33595]: pgmap v514: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-03T20:41:20.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:19 smithi181 ceph-mon[29801]: pgmap v515: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-05-03T20:41:20.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:19 smithi196 ceph-mon[32858]: pgmap v515: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-05-03T20:41:20.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:19 smithi204 ceph-mon[33595]: pgmap v515: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-05-03T20:41:22.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:21 smithi181 ceph-mon[29801]: pgmap v516: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-03T20:41:22.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:21 smithi196 ceph-mon[32858]: pgmap v516: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-03T20:41:22.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:21 smithi204 ceph-mon[33595]: pgmap v516: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-03T20:41:24.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:23 smithi181 ceph-mon[29801]: pgmap v517: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-05-03T20:41:24.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:23 smithi196 ceph-mon[32858]: pgmap v517: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-05-03T20:41:24.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:23 smithi204 ceph-mon[33595]: pgmap v517: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-05-03T20:41:25.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:24 smithi181 ceph-mon[29801]: pgmap v518: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-03T20:41:25.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:24 smithi196 ceph-mon[32858]: pgmap v518: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-03T20:41:25.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:24 smithi204 ceph-mon[33595]: pgmap v518: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-03T20:41:27.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:27 smithi196 ceph-mon[32858]: pgmap v519: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:27.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:27 smithi204 ceph-mon[33595]: pgmap v519: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:27.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:27 smithi181 ceph-mon[29801]: pgmap v519: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:29.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:29 smithi196 ceph-mon[32858]: pgmap v520: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:29.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:29 smithi204 ceph-mon[33595]: pgmap v520: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:29.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:29 smithi181 ceph-mon[29801]: pgmap v520: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:31.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:31 smithi196 ceph-mon[32858]: pgmap v521: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:41:31.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:31 smithi204 ceph-mon[33595]: pgmap v521: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:41:31.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:31 smithi181 ceph-mon[29801]: pgmap v521: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:41:33.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:33 smithi196 ceph-mon[32858]: pgmap v522: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:41:33.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:33 smithi204 ceph-mon[33595]: pgmap v522: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:41:33.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:33 smithi181 ceph-mon[29801]: pgmap v522: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:41:35.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:35 smithi196 ceph-mon[32858]: pgmap v523: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:41:35.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:35 smithi204 ceph-mon[33595]: pgmap v523: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:41:35.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:35 smithi181 ceph-mon[29801]: pgmap v523: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:41:37.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:37 smithi196 ceph-mon[32858]: pgmap v524: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:37.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:37 smithi204 ceph-mon[33595]: pgmap v524: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:37.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:37 smithi181 ceph-mon[29801]: pgmap v524: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:39.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:39 smithi196 ceph-mon[32858]: pgmap v525: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:39.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:39 smithi204 ceph-mon[33595]: pgmap v525: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:39.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:39 smithi181 ceph-mon[29801]: pgmap v525: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:41.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:41 smithi196 ceph-mon[32858]: pgmap v526: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:41:41.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:41 smithi204 ceph-mon[33595]: pgmap v526: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:41:41.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:41 smithi181 ceph-mon[29801]: pgmap v526: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:41:43.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:43 smithi196 ceph-mon[32858]: pgmap v527: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:41:43.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:43 smithi204 ceph-mon[33595]: pgmap v527: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:41:43.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:43 smithi181 ceph-mon[29801]: pgmap v527: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:41:45.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:45 smithi196 ceph-mon[32858]: pgmap v528: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:41:45.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:45 smithi204 ceph-mon[33595]: pgmap v528: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:41:45.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:45 smithi181 ceph-mon[29801]: pgmap v528: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:41:47.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:47 smithi196 ceph-mon[32858]: pgmap v529: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:47.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:47 smithi204 ceph-mon[33595]: pgmap v529: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:47.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:47 smithi181 ceph-mon[29801]: pgmap v529: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:49.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:49 smithi196 ceph-mon[32858]: pgmap v530: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:49.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:49 smithi204 ceph-mon[33595]: pgmap v530: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:49.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:49 smithi181 ceph-mon[29801]: pgmap v530: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:51.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:51 smithi196 ceph-mon[32858]: pgmap v531: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:41:51.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:51 smithi204 ceph-mon[33595]: pgmap v531: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:41:51.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:51 smithi181 ceph-mon[29801]: pgmap v531: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:41:53.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:53 smithi196 ceph-mon[32858]: pgmap v532: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:41:53.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:53 smithi204 ceph-mon[33595]: pgmap v532: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:41:53.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:53 smithi181 ceph-mon[29801]: pgmap v532: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:41:55.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:55 smithi196 ceph-mon[32858]: pgmap v533: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:41:55.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:55 smithi204 ceph-mon[33595]: pgmap v533: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:41:55.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:55 smithi181 ceph-mon[29801]: pgmap v533: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:41:57.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:57 smithi196 ceph-mon[32858]: pgmap v534: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:57.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:57 smithi204 ceph-mon[33595]: pgmap v534: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:57.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:57 smithi181 ceph-mon[29801]: pgmap v534: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:59.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:41:59 smithi196 ceph-mon[32858]: pgmap v535: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:59.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:41:59 smithi204 ceph-mon[33595]: pgmap v535: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:41:59.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:41:59 smithi181 ceph-mon[29801]: pgmap v535: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:01.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:01 smithi196 ceph-mon[32858]: pgmap v536: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:42:01.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:01 smithi204 ceph-mon[33595]: pgmap v536: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:42:01.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:01 smithi181 ceph-mon[29801]: pgmap v536: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:42:03.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:03 smithi196 ceph-mon[32858]: pgmap v537: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:42:03.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:03 smithi204 ceph-mon[33595]: pgmap v537: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:42:03.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:03 smithi181 ceph-mon[29801]: pgmap v537: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:42:05.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:05 smithi196 ceph-mon[32858]: pgmap v538: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:42:05.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:05 smithi204 ceph-mon[33595]: pgmap v538: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:42:05.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:05 smithi181 ceph-mon[29801]: pgmap v538: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:42:07.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:07 smithi196 ceph-mon[32858]: pgmap v539: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:07.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:07 smithi204 ceph-mon[33595]: pgmap v539: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:07.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:07 smithi181 ceph-mon[29801]: pgmap v539: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:09.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:09 smithi196 ceph-mon[32858]: pgmap v540: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:09.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:09 smithi204 ceph-mon[33595]: pgmap v540: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:09.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:09 smithi181 ceph-mon[29801]: pgmap v540: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:10.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:10 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:42:10.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:10 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:42:10.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:10 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:42:11.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:11 smithi196 ceph-mon[32858]: pgmap v541: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:42:11.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:11 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:42:11.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:11 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:42:11.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:11 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:42:11.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:11 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:42:11.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:11 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:42:11.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:11 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:42:11.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:11 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:42:11.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:11 smithi204 ceph-mon[33595]: pgmap v541: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:42:11.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:11 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:42:11.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:11 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:42:11.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:11 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:42:11.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:11 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:42:11.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:11 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:42:11.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:11 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:42:11.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:11 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:42:11.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:11 smithi181 ceph-mon[29801]: pgmap v541: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:42:11.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:11 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:42:11.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:11 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:42:11.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:11 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:42:11.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:11 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:42:11.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:11 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:42:11.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:11 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:42:11.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:11 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:42:12.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:12 smithi204 ceph-mon[33595]: Adjusting osd_memory_target on smithi181 to 176.1M 2024-05-03T20:42:12.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:12 smithi204 ceph-mon[33595]: Unable to set osd_memory_target on smithi181 to 184755916: error parsing value: Value '184755916' is below minimum 939524096 2024-05-03T20:42:12.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:12 smithi181 ceph-mon[29801]: Adjusting osd_memory_target on smithi181 to 176.1M 2024-05-03T20:42:12.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:12 smithi181 ceph-mon[29801]: Unable to set osd_memory_target on smithi181 to 184755916: error parsing value: Value '184755916' is below minimum 939524096 2024-05-03T20:42:13.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:12 smithi196 ceph-mon[32858]: Adjusting osd_memory_target on smithi181 to 176.1M 2024-05-03T20:42:13.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:12 smithi196 ceph-mon[32858]: Unable to set osd_memory_target on smithi181 to 184755916: error parsing value: Value '184755916' is below minimum 939524096 2024-05-03T20:42:13.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:13 smithi181 ceph-mon[29801]: pgmap v542: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:42:14.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:13 smithi196 ceph-mon[32858]: pgmap v542: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:42:14.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:13 smithi204 ceph-mon[33595]: pgmap v542: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:42:15.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:15 smithi204 ceph-mon[33595]: pgmap v543: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:42:15.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:15 smithi181 ceph-mon[29801]: pgmap v543: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:42:16.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:15 smithi196 ceph-mon[32858]: pgmap v543: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:42:17.908 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:17 smithi196 ceph-mon[32858]: pgmap v544: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:17.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:17 smithi181 ceph-mon[29801]: pgmap v544: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:18.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:17 smithi204 ceph-mon[33595]: pgmap v544: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:19.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:19 smithi181 ceph-mon[29801]: pgmap v545: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:20.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:19 smithi196 ceph-mon[32858]: pgmap v545: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:20.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:19 smithi204 ceph-mon[33595]: pgmap v545: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:21.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:21 smithi181 ceph-mon[29801]: pgmap v546: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:42:22.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:21 smithi196 ceph-mon[32858]: pgmap v546: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:42:22.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:21 smithi204 ceph-mon[33595]: pgmap v546: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:42:23.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:23 smithi181 ceph-mon[29801]: pgmap v547: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:42:24.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:23 smithi196 ceph-mon[32858]: pgmap v547: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:42:24.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:23 smithi204 ceph-mon[33595]: pgmap v547: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:42:25.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:25 smithi181 ceph-mon[29801]: pgmap v548: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:42:26.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:25 smithi196 ceph-mon[32858]: pgmap v548: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:42:26.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:25 smithi204 ceph-mon[33595]: pgmap v548: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:42:27.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:27 smithi181 ceph-mon[29801]: pgmap v549: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:28.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:27 smithi196 ceph-mon[32858]: pgmap v549: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:28.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:27 smithi204 ceph-mon[33595]: pgmap v549: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:29.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:29 smithi181 ceph-mon[29801]: pgmap v550: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:30.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:29 smithi196 ceph-mon[32858]: pgmap v550: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:30.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:29 smithi204 ceph-mon[33595]: pgmap v550: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:31.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:31 smithi181 ceph-mon[29801]: pgmap v551: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:42:32.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:31 smithi196 ceph-mon[32858]: pgmap v551: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:42:32.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:31 smithi204 ceph-mon[33595]: pgmap v551: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:42:33.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:33 smithi181 ceph-mon[29801]: pgmap v552: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:42:34.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:33 smithi196 ceph-mon[32858]: pgmap v552: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:42:34.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:33 smithi204 ceph-mon[33595]: pgmap v552: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:42:35.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:35 smithi181 ceph-mon[29801]: pgmap v553: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:42:36.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:35 smithi196 ceph-mon[32858]: pgmap v553: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:42:36.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:35 smithi204 ceph-mon[33595]: pgmap v553: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:42:37.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:37 smithi181 ceph-mon[29801]: pgmap v554: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:38.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:37 smithi196 ceph-mon[32858]: pgmap v554: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:38.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:37 smithi204 ceph-mon[33595]: pgmap v554: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:39.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:39 smithi181 ceph-mon[29801]: pgmap v555: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:40.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:39 smithi196 ceph-mon[32858]: pgmap v555: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:40.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:39 smithi204 ceph-mon[33595]: pgmap v555: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:41.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:41 smithi181 ceph-mon[29801]: pgmap v556: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:42:42.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:41 smithi196 ceph-mon[32858]: pgmap v556: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:42:42.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:41 smithi204 ceph-mon[33595]: pgmap v556: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:42:43.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:43 smithi181 ceph-mon[29801]: pgmap v557: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:42:44.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:43 smithi196 ceph-mon[32858]: pgmap v557: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:42:44.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:43 smithi204 ceph-mon[33595]: pgmap v557: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:42:45.915 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:45 smithi204 ceph-mon[33595]: pgmap v558: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:42:45.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:45 smithi181 ceph-mon[29801]: pgmap v558: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:42:46.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:45 smithi196 ceph-mon[32858]: pgmap v558: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:42:47.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:47 smithi181 ceph-mon[29801]: pgmap v559: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:48.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:47 smithi196 ceph-mon[32858]: pgmap v559: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:48.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:47 smithi204 ceph-mon[33595]: pgmap v559: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:49.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:49 smithi181 ceph-mon[29801]: pgmap v560: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:50.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:49 smithi196 ceph-mon[32858]: pgmap v560: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:50.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:49 smithi204 ceph-mon[33595]: pgmap v560: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:51.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:42:51.607+0000 7f420f7fe640 1 -- 172.21.15.181:0/3015235437 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f4208005850 con 0x7f4228102b20 2024-05-03T20:42:51.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:42:51.607+0000 7f422c81b640 1 -- 172.21.15.181:0/3015235437 <== mon.0 v2:172.21.15.181:3300/0 8 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f4208005850 con 0x7f4228102b20 2024-05-03T20:42:51.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:51 smithi181 ceph-mon[29801]: pgmap v561: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:42:52.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:51 smithi196 ceph-mon[32858]: pgmap v561: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:42:52.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:51 smithi204 ceph-mon[33595]: pgmap v561: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:42:53.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:53 smithi181 ceph-mon[29801]: pgmap v562: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:42:54.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:53 smithi196 ceph-mon[32858]: pgmap v562: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:42:54.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:53 smithi204 ceph-mon[33595]: pgmap v562: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:42:55.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:55 smithi181 ceph-mon[29801]: pgmap v563: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:42:56.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:55 smithi196 ceph-mon[32858]: pgmap v563: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:42:56.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:55 smithi204 ceph-mon[33595]: pgmap v563: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:42:57.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:57 smithi181 ceph-mon[29801]: pgmap v564: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:58.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:57 smithi196 ceph-mon[32858]: pgmap v564: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:58.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:57 smithi204 ceph-mon[33595]: pgmap v564: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:42:59.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:42:59 smithi181 ceph-mon[29801]: pgmap v565: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:00.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:42:59 smithi196 ceph-mon[32858]: pgmap v565: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:00.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:42:59 smithi204 ceph-mon[33595]: pgmap v565: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:01.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:01 smithi181 ceph-mon[29801]: pgmap v566: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:43:02.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:01 smithi196 ceph-mon[32858]: pgmap v566: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:43:02.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:01 smithi204 ceph-mon[33595]: pgmap v566: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:43:03.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:03 smithi181 ceph-mon[29801]: pgmap v567: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:43:04.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:03 smithi196 ceph-mon[32858]: pgmap v567: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:43:04.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:03 smithi204 ceph-mon[33595]: pgmap v567: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:43:05.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:05 smithi181 ceph-mon[29801]: pgmap v568: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:43:06.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:05 smithi196 ceph-mon[32858]: pgmap v568: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:43:06.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:05 smithi204 ceph-mon[33595]: pgmap v568: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:43:07.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:07 smithi181 ceph-mon[29801]: pgmap v569: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:08.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:07 smithi196 ceph-mon[32858]: pgmap v569: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:08.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:07 smithi204 ceph-mon[33595]: pgmap v569: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:09.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:09 smithi181 ceph-mon[29801]: pgmap v570: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:10.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:09 smithi196 ceph-mon[32858]: pgmap v570: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:10.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:09 smithi204 ceph-mon[33595]: pgmap v570: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:11.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:11 smithi181 ceph-mon[29801]: pgmap v571: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:43:11.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:11 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:43:12.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:11 smithi196 ceph-mon[32858]: pgmap v571: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:43:12.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:11 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:43:12.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:11 smithi204 ceph-mon[33595]: pgmap v571: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:43:12.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:11 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:43:12.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:43:12.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:43:12.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:43:12.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:43:12.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:43:12.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:43:12.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:43:12.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:43:12.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:12 smithi181 ceph-mon[29801]: Adjusting osd_memory_target on smithi196 to 176.2M 2024-05-03T20:43:12.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:12 smithi181 ceph-mon[29801]: Adjusting osd_memory_target on smithi204 to 176.1M 2024-05-03T20:43:12.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:12 smithi181 ceph-mon[29801]: Unable to set osd_memory_target on smithi196 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-05-03T20:43:12.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:12 smithi181 ceph-mon[29801]: Unable to set osd_memory_target on smithi204 to 184758067: error parsing value: Value '184758067' is below minimum 939524096 2024-05-03T20:43:12.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:43:12.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:43:12.964 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:43:13.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:43:13.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:43:13.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:43:13.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:43:13.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:43:13.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:43:13.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:43:13.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:43:13.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:12 smithi196 ceph-mon[32858]: Adjusting osd_memory_target on smithi196 to 176.2M 2024-05-03T20:43:13.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:12 smithi196 ceph-mon[32858]: Adjusting osd_memory_target on smithi204 to 176.1M 2024-05-03T20:43:13.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:12 smithi196 ceph-mon[32858]: Unable to set osd_memory_target on smithi196 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-05-03T20:43:13.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:12 smithi196 ceph-mon[32858]: Unable to set osd_memory_target on smithi204 to 184758067: error parsing value: Value '184758067' is below minimum 939524096 2024-05-03T20:43:13.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:43:13.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:43:13.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:43:13.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:43:13.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:43:13.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:43:13.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:43:13.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:43:13.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:43:13.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:43:13.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T20:43:13.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:12 smithi204 ceph-mon[33595]: Adjusting osd_memory_target on smithi196 to 176.2M 2024-05-03T20:43:13.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:12 smithi204 ceph-mon[33595]: Adjusting osd_memory_target on smithi204 to 176.1M 2024-05-03T20:43:13.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:12 smithi204 ceph-mon[33595]: Unable to set osd_memory_target on smithi196 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-05-03T20:43:13.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:12 smithi204 ceph-mon[33595]: Unable to set osd_memory_target on smithi204 to 184758067: error parsing value: Value '184758067' is below minimum 939524096 2024-05-03T20:43:13.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:43:13.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:43:13.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:43:14.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:13 smithi204 ceph-mon[33595]: pgmap v572: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:43:14.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:13 smithi181 ceph-mon[29801]: pgmap v572: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:43:14.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:13 smithi196 ceph-mon[32858]: pgmap v572: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:43:16.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:15 smithi204 ceph-mon[33595]: pgmap v573: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:43:16.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:15 smithi181 ceph-mon[29801]: pgmap v573: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:43:16.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:15 smithi196 ceph-mon[32858]: pgmap v573: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:43:18.158 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:17 smithi196 ceph-mon[32858]: pgmap v574: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:18.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:17 smithi181 ceph-mon[29801]: pgmap v574: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:18.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:17 smithi204 ceph-mon[33595]: pgmap v574: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:20.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:19 smithi181 ceph-mon[29801]: pgmap v575: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:20.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:19 smithi196 ceph-mon[32858]: pgmap v575: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:20.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:19 smithi204 ceph-mon[33595]: pgmap v575: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:22.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:21 smithi181 ceph-mon[29801]: pgmap v576: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:43:22.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:21 smithi196 ceph-mon[32858]: pgmap v576: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:43:22.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:21 smithi204 ceph-mon[33595]: pgmap v576: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:43:24.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:23 smithi181 ceph-mon[29801]: pgmap v577: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:43:24.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:23 smithi196 ceph-mon[32858]: pgmap v577: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:43:24.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:23 smithi204 ceph-mon[33595]: pgmap v577: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:43:26.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:25 smithi181 ceph-mon[29801]: pgmap v578: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:43:26.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:25 smithi196 ceph-mon[32858]: pgmap v578: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:43:26.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:25 smithi204 ceph-mon[33595]: pgmap v578: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:43:28.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:27 smithi181 ceph-mon[29801]: pgmap v579: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:28.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:27 smithi196 ceph-mon[32858]: pgmap v579: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:28.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:27 smithi204 ceph-mon[33595]: pgmap v579: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:30.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:29 smithi181 ceph-mon[29801]: pgmap v580: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:30.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:29 smithi196 ceph-mon[32858]: pgmap v580: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:30.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:29 smithi204 ceph-mon[33595]: pgmap v580: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:32.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:31 smithi181 ceph-mon[29801]: pgmap v581: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:43:32.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:31 smithi196 ceph-mon[32858]: pgmap v581: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:43:32.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:31 smithi204 ceph-mon[33595]: pgmap v581: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:43:34.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:33 smithi181 ceph-mon[29801]: pgmap v582: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:43:34.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:33 smithi196 ceph-mon[32858]: pgmap v582: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:43:34.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:33 smithi204 ceph-mon[33595]: pgmap v582: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:43:36.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:35 smithi181 ceph-mon[29801]: pgmap v583: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:43:36.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:35 smithi196 ceph-mon[32858]: pgmap v583: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:43:36.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:35 smithi204 ceph-mon[33595]: pgmap v583: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:43:38.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:37 smithi181 ceph-mon[29801]: pgmap v584: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:38.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:37 smithi196 ceph-mon[32858]: pgmap v584: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:38.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:37 smithi204 ceph-mon[33595]: pgmap v584: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:40.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:39 smithi181 ceph-mon[29801]: pgmap v585: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:40.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:39 smithi196 ceph-mon[32858]: pgmap v585: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:40.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:39 smithi204 ceph-mon[33595]: pgmap v585: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:42.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:41 smithi181 ceph-mon[29801]: pgmap v586: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:43:42.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:41 smithi196 ceph-mon[32858]: pgmap v586: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:43:42.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:41 smithi204 ceph-mon[33595]: pgmap v586: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:43:44.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:43 smithi181 ceph-mon[29801]: pgmap v587: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:43:44.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:43 smithi196 ceph-mon[32858]: pgmap v587: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:43:44.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:43 smithi204 ceph-mon[33595]: pgmap v587: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:43:46.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:45 smithi204 ceph-mon[33595]: pgmap v588: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:43:46.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:45 smithi181 ceph-mon[29801]: pgmap v588: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:43:46.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:45 smithi196 ceph-mon[32858]: pgmap v588: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:43:48.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:47 smithi181 ceph-mon[29801]: pgmap v589: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:48.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:47 smithi196 ceph-mon[32858]: pgmap v589: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:48.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:47 smithi204 ceph-mon[33595]: pgmap v589: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:50.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:49 smithi181 ceph-mon[29801]: pgmap v590: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:50.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:49 smithi196 ceph-mon[32858]: pgmap v590: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:50.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:49 smithi204 ceph-mon[33595]: pgmap v590: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:52.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:51 smithi181 ceph-mon[29801]: pgmap v591: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:43:52.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:51 smithi196 ceph-mon[32858]: pgmap v591: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:43:52.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:51 smithi204 ceph-mon[33595]: pgmap v591: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:43:53.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:52 smithi181 ceph-mon[29801]: pgmap v592: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:43:53.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:52 smithi196 ceph-mon[32858]: pgmap v592: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:43:53.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:52 smithi204 ceph-mon[33595]: pgmap v592: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:43:55.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:55 smithi196 ceph-mon[32858]: pgmap v593: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:43:55.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:55 smithi204 ceph-mon[33595]: pgmap v593: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:43:55.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:55 smithi181 ceph-mon[29801]: pgmap v593: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:43:57.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:57 smithi196 ceph-mon[32858]: pgmap v594: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:57.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:57 smithi204 ceph-mon[33595]: pgmap v594: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:57.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:57 smithi181 ceph-mon[29801]: pgmap v594: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:59.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:43:59 smithi196 ceph-mon[32858]: pgmap v595: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:59.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:43:59 smithi204 ceph-mon[33595]: pgmap v595: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:43:59.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:43:59 smithi181 ceph-mon[29801]: pgmap v595: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:01.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:01 smithi196 ceph-mon[32858]: pgmap v596: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:44:01.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:01 smithi204 ceph-mon[33595]: pgmap v596: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:44:01.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:01 smithi181 ceph-mon[29801]: pgmap v596: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:44:03.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:03 smithi196 ceph-mon[32858]: pgmap v597: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:44:03.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:03 smithi204 ceph-mon[33595]: pgmap v597: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:44:03.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:03 smithi181 ceph-mon[29801]: pgmap v597: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:44:05.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:05 smithi196 ceph-mon[32858]: pgmap v598: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:44:05.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:05 smithi204 ceph-mon[33595]: pgmap v598: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:44:05.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:05 smithi181 ceph-mon[29801]: pgmap v598: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:44:07.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:07 smithi196 ceph-mon[32858]: pgmap v599: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:07.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:07 smithi204 ceph-mon[33595]: pgmap v599: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:07.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:07 smithi181 ceph-mon[29801]: pgmap v599: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:09.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:09 smithi196 ceph-mon[32858]: pgmap v600: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:09.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:09 smithi204 ceph-mon[33595]: pgmap v600: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:09.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:09 smithi181 ceph-mon[29801]: pgmap v600: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:11.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:11 smithi196 ceph-mon[32858]: pgmap v601: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:44:11.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:11 smithi204 ceph-mon[33595]: pgmap v601: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:44:11.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:11 smithi181 ceph-mon[29801]: pgmap v601: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:44:12.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:12 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:44:12.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:12 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:44:12.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:12 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:44:13.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:13 smithi181 ceph-mon[29801]: pgmap v602: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:44:13.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:13 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:44:13.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:13 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:44:13.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:13 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:44:13.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:13 smithi196 ceph-mon[32858]: pgmap v602: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:44:13.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:13 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:44:13.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:13 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:44:13.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:13 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:44:13.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:13 smithi204 ceph-mon[33595]: pgmap v602: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:44:13.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:13 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:44:13.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:13 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:44:13.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:13 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:44:15.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:15 smithi181 ceph-mon[29801]: pgmap v603: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:44:15.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:15 smithi196 ceph-mon[32858]: pgmap v603: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:44:15.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:15 smithi204 ceph-mon[33595]: pgmap v603: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:44:17.415 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:17 smithi204 ceph-mon[33595]: pgmap v604: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:17.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:17 smithi181 ceph-mon[29801]: pgmap v604: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:17.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:17 smithi196 ceph-mon[32858]: pgmap v604: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:19.408 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:19 smithi196 ceph-mon[32858]: pgmap v605: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:19.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:19 smithi181 ceph-mon[29801]: pgmap v605: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:19.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:19 smithi204 ceph-mon[33595]: pgmap v605: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:21.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:21 smithi181 ceph-mon[29801]: pgmap v606: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:44:21.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:21 smithi196 ceph-mon[32858]: pgmap v606: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:44:21.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:21 smithi204 ceph-mon[33595]: pgmap v606: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:44:23.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:23 smithi181 ceph-mon[29801]: pgmap v607: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:44:23.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:23 smithi196 ceph-mon[32858]: pgmap v607: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:44:23.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:23 smithi204 ceph-mon[33595]: pgmap v607: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:44:25.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:25 smithi181 ceph-mon[29801]: pgmap v608: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:44:25.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:25 smithi196 ceph-mon[32858]: pgmap v608: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:44:25.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:25 smithi204 ceph-mon[33595]: pgmap v608: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:44:27.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:27 smithi181 ceph-mon[29801]: pgmap v609: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:27.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:27 smithi196 ceph-mon[32858]: pgmap v609: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:27.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:27 smithi204 ceph-mon[33595]: pgmap v609: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:29.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:29 smithi181 ceph-mon[29801]: pgmap v610: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:29.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:29 smithi196 ceph-mon[32858]: pgmap v610: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:29.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:29 smithi204 ceph-mon[33595]: pgmap v610: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:31.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:31 smithi181 ceph-mon[29801]: pgmap v611: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:44:31.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:31 smithi196 ceph-mon[32858]: pgmap v611: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:44:31.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:31 smithi204 ceph-mon[33595]: pgmap v611: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:44:33.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:33 smithi181 ceph-mon[29801]: pgmap v612: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:44:33.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:33 smithi196 ceph-mon[32858]: pgmap v612: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:44:33.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:33 smithi204 ceph-mon[33595]: pgmap v612: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:44:35.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:35 smithi181 ceph-mon[29801]: pgmap v613: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:44:35.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:35 smithi196 ceph-mon[32858]: pgmap v613: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:44:35.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:35 smithi204 ceph-mon[33595]: pgmap v613: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:44:37.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:37 smithi181 ceph-mon[29801]: pgmap v614: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:37.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:37 smithi196 ceph-mon[32858]: pgmap v614: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:37.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:37 smithi204 ceph-mon[33595]: pgmap v614: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:39.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:39 smithi181 ceph-mon[29801]: pgmap v615: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:39.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:39 smithi196 ceph-mon[32858]: pgmap v615: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:39.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:39 smithi204 ceph-mon[33595]: pgmap v615: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:41.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:41 smithi181 ceph-mon[29801]: pgmap v616: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:44:41.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:41 smithi196 ceph-mon[32858]: pgmap v616: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:44:41.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:41 smithi204 ceph-mon[33595]: pgmap v616: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:44:43.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:43 smithi181 ceph-mon[29801]: pgmap v617: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:44:43.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:43 smithi196 ceph-mon[32858]: pgmap v617: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:44:43.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:43 smithi204 ceph-mon[33595]: pgmap v617: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:44:45.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:45 smithi181 ceph-mon[29801]: pgmap v618: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:44:45.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:45 smithi196 ceph-mon[32858]: pgmap v618: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:44:45.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:45 smithi204 ceph-mon[33595]: pgmap v618: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:44:47.414 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:47 smithi204 ceph-mon[33595]: pgmap v619: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:47.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:47 smithi181 ceph-mon[29801]: pgmap v619: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:47.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:47 smithi196 ceph-mon[32858]: pgmap v619: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:49.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:49 smithi181 ceph-mon[29801]: pgmap v620: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:49.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:49 smithi196 ceph-mon[32858]: pgmap v620: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:49.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:49 smithi204 ceph-mon[33595]: pgmap v620: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:51.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:51 smithi181 ceph-mon[29801]: pgmap v621: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:44:51.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:51 smithi196 ceph-mon[32858]: pgmap v621: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:44:51.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:51 smithi204 ceph-mon[33595]: pgmap v621: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:44:53.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:53 smithi181 ceph-mon[29801]: pgmap v622: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:44:53.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:53 smithi196 ceph-mon[32858]: pgmap v622: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:44:53.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:53 smithi204 ceph-mon[33595]: pgmap v622: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:44:55.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:55 smithi181 ceph-mon[29801]: pgmap v623: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:44:55.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:55 smithi196 ceph-mon[32858]: pgmap v623: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:44:55.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:55 smithi204 ceph-mon[33595]: pgmap v623: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:44:57.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:57 smithi181 ceph-mon[29801]: pgmap v624: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:57.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:57 smithi196 ceph-mon[32858]: pgmap v624: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:57.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:57 smithi204 ceph-mon[33595]: pgmap v624: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:59.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:44:59 smithi181 ceph-mon[29801]: pgmap v625: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:59.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:44:59 smithi196 ceph-mon[32858]: pgmap v625: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:44:59.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:44:59 smithi204 ceph-mon[33595]: pgmap v625: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:01.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:01 smithi181 ceph-mon[29801]: pgmap v626: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:45:01.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:01 smithi196 ceph-mon[32858]: pgmap v626: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:45:01.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:01 smithi204 ceph-mon[33595]: pgmap v626: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:45:03.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:03 smithi181 ceph-mon[29801]: pgmap v627: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:45:03.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:03 smithi196 ceph-mon[32858]: pgmap v627: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:45:03.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:03 smithi204 ceph-mon[33595]: pgmap v627: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:45:05.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:05 smithi181 ceph-mon[29801]: pgmap v628: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:45:05.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:05 smithi196 ceph-mon[32858]: pgmap v628: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:45:05.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:05 smithi204 ceph-mon[33595]: pgmap v628: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:45:06.624 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:45:06.626+0000 7f420f7fe640 1 -- 172.21.15.181:0/3015235437 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f4208005b90 con 0x7f4228102b20 2024-05-03T20:45:06.625 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:45:06.627+0000 7f422c81b640 1 -- 172.21.15.181:0/3015235437 <== mon.0 v2:172.21.15.181:3300/0 9 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f4208005b90 con 0x7f4228102b20 2024-05-03T20:45:07.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:07 smithi181 ceph-mon[29801]: pgmap v629: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:07.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:07 smithi196 ceph-mon[32858]: pgmap v629: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:07.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:07 smithi204 ceph-mon[33595]: pgmap v629: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:09.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:09 smithi181 ceph-mon[29801]: pgmap v630: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:09.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:09 smithi196 ceph-mon[32858]: pgmap v630: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:09.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:09 smithi204 ceph-mon[33595]: pgmap v630: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:11.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:11 smithi181 ceph-mon[29801]: pgmap v631: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:45:11.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:11 smithi196 ceph-mon[32858]: pgmap v631: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:45:11.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:11 smithi204 ceph-mon[33595]: pgmap v631: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:45:13.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:13 smithi181 ceph-mon[29801]: pgmap v632: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:45:13.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:13 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:45:13.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:13 smithi196 ceph-mon[32858]: pgmap v632: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:45:13.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:13 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:45:13.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:13 smithi204 ceph-mon[33595]: pgmap v632: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:45:13.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:13 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:45:14.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:14 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:45:14.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:14 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:45:14.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:14 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:45:14.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:14 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:45:14.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:14 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:45:14.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:14 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:45:14.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:14 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:45:14.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:14 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:45:14.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:14 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:45:15.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:15 smithi196 ceph-mon[32858]: pgmap v633: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:45:15.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:15 smithi204 ceph-mon[33595]: pgmap v633: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:45:15.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:15 smithi181 ceph-mon[29801]: pgmap v633: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:45:17.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:17 smithi196 ceph-mon[32858]: pgmap v634: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:17.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:17 smithi204 ceph-mon[33595]: pgmap v634: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:17.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:17 smithi181 ceph-mon[29801]: pgmap v634: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:19.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:19 smithi196 ceph-mon[32858]: pgmap v635: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:19.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:19 smithi204 ceph-mon[33595]: pgmap v635: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:19.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:19 smithi181 ceph-mon[29801]: pgmap v635: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:21.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:21 smithi196 ceph-mon[32858]: pgmap v636: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:45:21.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:21 smithi204 ceph-mon[33595]: pgmap v636: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:45:21.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:21 smithi181 ceph-mon[29801]: pgmap v636: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:45:23.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:23 smithi196 ceph-mon[32858]: pgmap v637: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:45:23.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:23 smithi204 ceph-mon[33595]: pgmap v637: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:45:23.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:23 smithi181 ceph-mon[29801]: pgmap v637: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:45:25.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:25 smithi196 ceph-mon[32858]: pgmap v638: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:45:25.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:25 smithi204 ceph-mon[33595]: pgmap v638: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:45:25.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:25 smithi181 ceph-mon[29801]: pgmap v638: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:45:27.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:27 smithi196 ceph-mon[32858]: pgmap v639: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:27.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:27 smithi204 ceph-mon[33595]: pgmap v639: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:27.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:27 smithi181 ceph-mon[29801]: pgmap v639: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:29.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:29 smithi196 ceph-mon[32858]: pgmap v640: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:29.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:29 smithi204 ceph-mon[33595]: pgmap v640: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:29.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:29 smithi181 ceph-mon[29801]: pgmap v640: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:31.765 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:31 smithi181 ceph-mon[29801]: pgmap v641: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:45:31.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:31 smithi196 ceph-mon[32858]: pgmap v641: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:45:31.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:31 smithi204 ceph-mon[33595]: pgmap v641: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:45:33.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:33 smithi196 ceph-mon[32858]: pgmap v642: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:45:33.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:33 smithi204 ceph-mon[33595]: pgmap v642: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:45:33.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:33 smithi181 ceph-mon[29801]: pgmap v642: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:45:35.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:35 smithi196 ceph-mon[32858]: pgmap v643: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:45:35.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:35 smithi204 ceph-mon[33595]: pgmap v643: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:45:35.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:35 smithi181 ceph-mon[29801]: pgmap v643: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:45:37.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:37 smithi196 ceph-mon[32858]: pgmap v644: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:37.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:37 smithi204 ceph-mon[33595]: pgmap v644: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:37.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:37 smithi181 ceph-mon[29801]: pgmap v644: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:39.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:39 smithi196 ceph-mon[32858]: pgmap v645: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:39.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:39 smithi204 ceph-mon[33595]: pgmap v645: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:39.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:39 smithi181 ceph-mon[29801]: pgmap v645: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:41.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:41 smithi196 ceph-mon[32858]: pgmap v646: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:45:41.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:41 smithi204 ceph-mon[33595]: pgmap v646: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:45:41.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:41 smithi181 ceph-mon[29801]: pgmap v646: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:45:43.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:43 smithi204 ceph-mon[33595]: pgmap v647: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:45:43.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:43 smithi181 ceph-mon[29801]: pgmap v647: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:45:44.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:43 smithi196 ceph-mon[32858]: pgmap v647: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:45:45.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:45 smithi204 ceph-mon[33595]: pgmap v648: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:45:45.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:45 smithi181 ceph-mon[29801]: pgmap v648: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:45:46.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:45 smithi196 ceph-mon[32858]: pgmap v648: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:45:47.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:47 smithi181 ceph-mon[29801]: pgmap v649: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:48.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:47 smithi196 ceph-mon[32858]: pgmap v649: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:48.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:47 smithi204 ceph-mon[33595]: pgmap v649: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:49.908 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:49 smithi196 ceph-mon[32858]: pgmap v650: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:49.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:49 smithi181 ceph-mon[29801]: pgmap v650: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:50.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:49 smithi204 ceph-mon[33595]: pgmap v650: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:51.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:45:51.632+0000 7f420f7fe640 1 -- 172.21.15.181:0/3015235437 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f4208005ee0 con 0x7f4228102b20 2024-05-03T20:45:51.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:45:51.632+0000 7f422c81b640 1 -- 172.21.15.181:0/3015235437 <== mon.0 v2:172.21.15.181:3300/0 10 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f4208005ee0 con 0x7f4228102b20 2024-05-03T20:45:51.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:51 smithi181 ceph-mon[29801]: pgmap v651: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:45:52.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:51 smithi196 ceph-mon[32858]: pgmap v651: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:45:52.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:51 smithi204 ceph-mon[33595]: pgmap v651: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:45:53.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:53 smithi181 ceph-mon[29801]: pgmap v652: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:45:54.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:53 smithi196 ceph-mon[32858]: pgmap v652: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:45:54.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:53 smithi204 ceph-mon[33595]: pgmap v652: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:45:55.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:55 smithi181 ceph-mon[29801]: pgmap v653: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:45:56.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:55 smithi196 ceph-mon[32858]: pgmap v653: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:45:56.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:55 smithi204 ceph-mon[33595]: pgmap v653: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:45:57.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:57 smithi181 ceph-mon[29801]: pgmap v654: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:58.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:57 smithi196 ceph-mon[32858]: pgmap v654: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:58.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:57 smithi204 ceph-mon[33595]: pgmap v654: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:45:59.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:45:59 smithi181 ceph-mon[29801]: pgmap v655: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:00.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:45:59 smithi196 ceph-mon[32858]: pgmap v655: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:00.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:45:59 smithi204 ceph-mon[33595]: pgmap v655: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:01.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:01 smithi181 ceph-mon[29801]: pgmap v656: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:46:02.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:01 smithi196 ceph-mon[32858]: pgmap v656: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:46:02.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:01 smithi204 ceph-mon[33595]: pgmap v656: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:46:03.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:03 smithi181 ceph-mon[29801]: pgmap v657: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:46:04.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:03 smithi196 ceph-mon[32858]: pgmap v657: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:46:04.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:03 smithi204 ceph-mon[33595]: pgmap v657: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:46:05.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:05 smithi181 ceph-mon[29801]: pgmap v658: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:46:06.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:05 smithi196 ceph-mon[32858]: pgmap v658: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:46:06.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:05 smithi204 ceph-mon[33595]: pgmap v658: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:46:07.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:07 smithi181 ceph-mon[29801]: pgmap v659: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:08.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:07 smithi196 ceph-mon[32858]: pgmap v659: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:08.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:07 smithi204 ceph-mon[33595]: pgmap v659: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:09.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:09 smithi181 ceph-mon[29801]: pgmap v660: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:10.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:09 smithi196 ceph-mon[32858]: pgmap v660: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:10.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:09 smithi204 ceph-mon[33595]: pgmap v660: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:11.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:11 smithi181 ceph-mon[29801]: pgmap v661: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:46:12.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:11 smithi196 ceph-mon[32858]: pgmap v661: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:46:12.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:11 smithi204 ceph-mon[33595]: pgmap v661: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:46:13.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:13 smithi181 ceph-mon[29801]: pgmap v662: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:46:13.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:13 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:46:14.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:13 smithi196 ceph-mon[32858]: pgmap v662: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:46:14.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:13 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:46:14.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:13 smithi204 ceph-mon[33595]: pgmap v662: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:46:14.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:13 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:46:15.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:15 smithi181 ceph-mon[29801]: pgmap v663: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:46:15.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:15 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:46:15.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:15 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:46:15.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:15 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:46:15.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:15 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:46:15.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:15 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:46:15.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:15 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:46:16.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:15 smithi196 ceph-mon[32858]: pgmap v663: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:46:16.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:15 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:46:16.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:15 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:46:16.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:15 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:46:16.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:15 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:46:16.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:15 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:46:16.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:15 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:46:16.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:15 smithi204 ceph-mon[33595]: pgmap v663: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:46:16.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:15 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:46:16.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:15 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:46:16.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:15 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:46:16.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:15 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:46:16.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:15 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:46:16.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:15 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:46:16.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:16 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:46:16.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:16 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:46:16.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:16 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:46:17.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:16 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:46:17.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:16 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:46:17.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:16 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:46:17.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:16 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:46:17.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:16 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:46:17.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:16 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:46:17.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:17 smithi181 ceph-mon[29801]: pgmap v664: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:18.026 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:17 smithi204 ceph-mon[33595]: pgmap v664: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:18.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:17 smithi196 ceph-mon[32858]: pgmap v664: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:19.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:19 smithi181 ceph-mon[29801]: pgmap v665: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:20.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:19 smithi196 ceph-mon[32858]: pgmap v665: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:20.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:19 smithi204 ceph-mon[33595]: pgmap v665: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:21.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:21 smithi181 ceph-mon[29801]: pgmap v666: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:46:22.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:21 smithi196 ceph-mon[32858]: pgmap v666: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:46:22.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:21 smithi204 ceph-mon[33595]: pgmap v666: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:46:23.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:23 smithi181 ceph-mon[29801]: pgmap v667: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:46:24.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:23 smithi196 ceph-mon[32858]: pgmap v667: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:46:24.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:23 smithi204 ceph-mon[33595]: pgmap v667: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:46:25.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:25 smithi181 ceph-mon[29801]: pgmap v668: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:46:26.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:25 smithi196 ceph-mon[32858]: pgmap v668: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:46:26.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:25 smithi204 ceph-mon[33595]: pgmap v668: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:46:27.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:27 smithi181 ceph-mon[29801]: pgmap v669: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:28.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:27 smithi196 ceph-mon[32858]: pgmap v669: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:28.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:27 smithi204 ceph-mon[33595]: pgmap v669: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:29.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:29 smithi181 ceph-mon[29801]: pgmap v670: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:30.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:29 smithi196 ceph-mon[32858]: pgmap v670: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:30.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:29 smithi204 ceph-mon[33595]: pgmap v670: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:31.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:31 smithi181 ceph-mon[29801]: pgmap v671: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:46:32.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:31 smithi196 ceph-mon[32858]: pgmap v671: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:46:32.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:31 smithi204 ceph-mon[33595]: pgmap v671: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:46:33.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:33 smithi181 ceph-mon[29801]: pgmap v672: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:46:34.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:33 smithi196 ceph-mon[32858]: pgmap v672: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:46:34.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:33 smithi204 ceph-mon[33595]: pgmap v672: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:46:35.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:35 smithi181 ceph-mon[29801]: pgmap v673: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:46:36.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:35 smithi196 ceph-mon[32858]: pgmap v673: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:46:36.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:35 smithi204 ceph-mon[33595]: pgmap v673: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:46:37.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:37 smithi181 ceph-mon[29801]: pgmap v674: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:38.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:37 smithi196 ceph-mon[32858]: pgmap v674: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:38.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:37 smithi204 ceph-mon[33595]: pgmap v674: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:39.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:39 smithi181 ceph-mon[29801]: pgmap v675: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:40.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:39 smithi196 ceph-mon[32858]: pgmap v675: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:40.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:39 smithi204 ceph-mon[33595]: pgmap v675: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:41.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:41 smithi181 ceph-mon[29801]: pgmap v676: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:46:42.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:41 smithi196 ceph-mon[32858]: pgmap v676: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:46:42.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:41 smithi204 ceph-mon[33595]: pgmap v676: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:46:43.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:43 smithi181 ceph-mon[29801]: pgmap v677: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:46:44.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:43 smithi196 ceph-mon[32858]: pgmap v677: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:46:44.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:43 smithi204 ceph-mon[33595]: pgmap v677: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:46:45.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:45 smithi181 ceph-mon[29801]: pgmap v678: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:46:46.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:45 smithi196 ceph-mon[32858]: pgmap v678: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:46:46.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:45 smithi204 ceph-mon[33595]: pgmap v678: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:46:47.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:47 smithi181 ceph-mon[29801]: pgmap v679: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:48.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:47 smithi196 ceph-mon[32858]: pgmap v679: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:48.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:47 smithi204 ceph-mon[33595]: pgmap v679: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:49.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:49 smithi181 ceph-mon[29801]: pgmap v680: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:50.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:49 smithi196 ceph-mon[32858]: pgmap v680: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:50.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:49 smithi204 ceph-mon[33595]: pgmap v680: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:51.966 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:51 smithi181 ceph-mon[29801]: pgmap v681: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:46:52.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:51 smithi196 ceph-mon[32858]: pgmap v681: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:46:52.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:51 smithi204 ceph-mon[33595]: pgmap v681: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:46:54.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:53 smithi196 ceph-mon[32858]: pgmap v682: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:46:54.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:53 smithi204 ceph-mon[33595]: pgmap v682: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:46:54.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:53 smithi181 ceph-mon[29801]: pgmap v682: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:46:56.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:55 smithi196 ceph-mon[32858]: pgmap v683: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:46:56.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:55 smithi204 ceph-mon[33595]: pgmap v683: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:46:56.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:55 smithi181 ceph-mon[29801]: pgmap v683: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:46:58.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:57 smithi196 ceph-mon[32858]: pgmap v684: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:58.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:57 smithi204 ceph-mon[33595]: pgmap v684: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:46:58.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:57 smithi181 ceph-mon[29801]: pgmap v684: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:47:00.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:46:59 smithi196 ceph-mon[32858]: pgmap v685: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:47:00.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:46:59 smithi204 ceph-mon[33595]: pgmap v685: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:47:00.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:46:59 smithi181 ceph-mon[29801]: pgmap v685: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:47:02.015 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:01 smithi181 ceph-mon[29801]: pgmap v686: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:47:02.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:01 smithi196 ceph-mon[32858]: pgmap v686: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:47:02.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:01 smithi204 ceph-mon[33595]: pgmap v686: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:47:04.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:03 smithi196 ceph-mon[32858]: pgmap v687: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:47:04.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:03 smithi204 ceph-mon[33595]: pgmap v687: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:47:04.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:03 smithi181 ceph-mon[29801]: pgmap v687: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:47:06.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:05 smithi196 ceph-mon[32858]: pgmap v688: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:47:06.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:05 smithi204 ceph-mon[33595]: pgmap v688: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:47:06.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:05 smithi181 ceph-mon[29801]: pgmap v688: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:47:08.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:07 smithi196 ceph-mon[32858]: pgmap v689: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:47:08.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:07 smithi204 ceph-mon[33595]: pgmap v689: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:47:08.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:07 smithi181 ceph-mon[29801]: pgmap v689: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:47:10.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:09 smithi196 ceph-mon[32858]: pgmap v690: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:47:10.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:09 smithi204 ceph-mon[33595]: pgmap v690: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:47:10.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:09 smithi181 ceph-mon[29801]: pgmap v690: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:47:12.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:11 smithi196 ceph-mon[32858]: pgmap v691: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:47:12.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:11 smithi204 ceph-mon[33595]: pgmap v691: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:47:12.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:11 smithi181 ceph-mon[29801]: pgmap v691: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:47:14.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:13 smithi196 ceph-mon[32858]: pgmap v692: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:47:14.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:13 smithi204 ceph-mon[33595]: pgmap v692: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:47:14.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:13 smithi181 ceph-mon[29801]: pgmap v692: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:47:16.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:15 smithi196 ceph-mon[32858]: pgmap v693: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:47:16.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:15 smithi204 ceph-mon[33595]: pgmap v693: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:47:16.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:15 smithi181 ceph-mon[29801]: pgmap v693: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:47:17.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:16 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:47:17.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:16 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:47:17.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:16 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:47:17.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:16 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:47:17.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:16 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:47:17.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:16 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:47:17.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:16 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:47:17.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:16 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:47:17.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:16 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:47:17.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:16 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:47:17.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:16 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:47:17.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:16 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:47:17.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:16 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:47:17.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:16 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:47:17.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:16 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:47:18.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:17 smithi196 ceph-mon[32858]: pgmap v694: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:47:18.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:17 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:47:18.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:17 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:47:18.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:17 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:47:18.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:17 smithi204 ceph-mon[33595]: pgmap v694: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:47:18.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:17 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:47:18.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:17 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:47:18.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:17 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:47:18.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:17 smithi181 ceph-mon[29801]: pgmap v694: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:47:18.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:17 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:47:18.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:17 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:47:18.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:17 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:47:20.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:19 smithi196 ceph-mon[32858]: pgmap v695: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:47:20.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:19 smithi204 ceph-mon[33595]: pgmap v695: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:47:20.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:19 smithi181 ceph-mon[29801]: pgmap v695: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:47:22.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:21 smithi196 ceph-mon[32858]: pgmap v696: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:47:22.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:21 smithi204 ceph-mon[33595]: pgmap v696: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:47:22.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:21 smithi181 ceph-mon[29801]: pgmap v696: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-03T20:47:24.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:23 smithi196 ceph-mon[32858]: pgmap v697: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:47:24.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:23 smithi204 ceph-mon[33595]: pgmap v697: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:47:24.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:23 smithi181 ceph-mon[29801]: pgmap v697: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:47:26.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:25 smithi196 ceph-mon[32858]: pgmap v698: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:47:26.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:25 smithi204 ceph-mon[33595]: pgmap v698: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:47:26.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:25 smithi181 ceph-mon[29801]: pgmap v698: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:47:28.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:27 smithi196 ceph-mon[32858]: pgmap v699: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:47:28.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:27 smithi204 ceph-mon[33595]: pgmap v699: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:47:28.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:27 smithi181 ceph-mon[29801]: pgmap v699: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:47:30.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:29 smithi196 ceph-mon[32858]: pgmap v700: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:47:30.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:29 smithi204 ceph-mon[33595]: pgmap v700: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:47:30.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:29 smithi181 ceph-mon[29801]: pgmap v700: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:47:32.105 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:31 smithi196 ceph-mon[32858]: pgmap v701: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:47:32.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:31 smithi204 ceph-mon[33595]: pgmap v701: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:47:32.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:31 smithi181 ceph-mon[29801]: pgmap v701: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:47:34.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:33 smithi196 ceph-mon[32858]: pgmap v702: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:47:34.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:33 smithi204 ceph-mon[33595]: pgmap v702: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:47:34.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:33 smithi181 ceph-mon[29801]: pgmap v702: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:47:36.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:35 smithi196 ceph-mon[32858]: pgmap v703: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:47:36.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:35 smithi204 ceph-mon[33595]: pgmap v703: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:47:36.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:35 smithi181 ceph-mon[29801]: pgmap v703: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:47:38.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:37 smithi196 ceph-mon[32858]: pgmap v704: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:47:38.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:37 smithi204 ceph-mon[33595]: pgmap v704: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:47:38.216 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:37 smithi181 ceph-mon[29801]: pgmap v704: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:47:40.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:39 smithi196 ceph-mon[32858]: pgmap v705: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:47:40.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:39 smithi204 ceph-mon[33595]: pgmap v705: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:47:40.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:39 smithi181 ceph-mon[29801]: pgmap v705: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:47:42.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:41 smithi196 ceph-mon[32858]: pgmap v706: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:47:42.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:41 smithi204 ceph-mon[33595]: pgmap v706: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:47:42.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:41 smithi181 ceph-mon[29801]: pgmap v706: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:47:44.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:43 smithi196 ceph-mon[32858]: pgmap v707: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:47:44.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:43 smithi204 ceph-mon[33595]: pgmap v707: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:47:44.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:43 smithi181 ceph-mon[29801]: pgmap v707: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:47:46.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:45 smithi196 ceph-mon[32858]: pgmap v708: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:47:46.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:45 smithi204 ceph-mon[33595]: pgmap v708: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:47:46.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:45 smithi181 ceph-mon[29801]: pgmap v708: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:47:48.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:47 smithi196 ceph-mon[32858]: pgmap v709: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:47:48.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:47 smithi204 ceph-mon[33595]: pgmap v709: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:47:48.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:47 smithi181 ceph-mon[29801]: pgmap v709: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:47:50.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:49 smithi196 ceph-mon[32858]: pgmap v710: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:47:50.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:49 smithi204 ceph-mon[33595]: pgmap v710: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:47:50.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:49 smithi181 ceph-mon[29801]: pgmap v710: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:47:52.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:51 smithi196 ceph-mon[32858]: pgmap v711: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:47:52.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:51 smithi204 ceph-mon[33595]: pgmap v711: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:47:52.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:51 smithi181 ceph-mon[29801]: pgmap v711: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:47:54.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:53 smithi196 ceph-mon[32858]: pgmap v712: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:47:54.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:53 smithi204 ceph-mon[33595]: pgmap v712: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:47:54.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:53 smithi181 ceph-mon[29801]: pgmap v712: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:47:56.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:55 smithi196 ceph-mon[32858]: pgmap v713: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:47:56.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:55 smithi204 ceph-mon[33595]: pgmap v713: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:47:56.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:55 smithi181 ceph-mon[29801]: pgmap v713: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:47:58.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:57 smithi196 ceph-mon[32858]: pgmap v714: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:47:58.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:57 smithi204 ceph-mon[33595]: pgmap v714: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:47:58.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:57 smithi181 ceph-mon[29801]: pgmap v714: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:48:00.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:47:59 smithi196 ceph-mon[32858]: pgmap v715: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:48:00.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:47:59 smithi204 ceph-mon[33595]: pgmap v715: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:48:00.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:47:59 smithi181 ceph-mon[29801]: pgmap v715: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:48:02.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:01 smithi196 ceph-mon[32858]: pgmap v716: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:48:02.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:01 smithi204 ceph-mon[33595]: pgmap v716: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:48:02.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:01 smithi181 ceph-mon[29801]: pgmap v716: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:48:04.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:03 smithi196 ceph-mon[32858]: pgmap v717: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:48:04.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:03 smithi204 ceph-mon[33595]: pgmap v717: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:48:04.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:03 smithi181 ceph-mon[29801]: pgmap v717: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:48:06.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:05 smithi196 ceph-mon[32858]: pgmap v718: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:48:06.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:05 smithi204 ceph-mon[33595]: pgmap v718: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:48:06.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:05 smithi181 ceph-mon[29801]: pgmap v718: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:48:08.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:07 smithi196 ceph-mon[32858]: pgmap v719: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:48:08.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:07 smithi204 ceph-mon[33595]: pgmap v719: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:48:08.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:07 smithi181 ceph-mon[29801]: pgmap v719: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:48:10.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:09 smithi196 ceph-mon[32858]: pgmap v720: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:48:10.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:09 smithi204 ceph-mon[33595]: pgmap v720: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:48:10.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:09 smithi181 ceph-mon[29801]: pgmap v720: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:48:12.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:11 smithi196 ceph-mon[32858]: pgmap v721: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:48:12.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:11 smithi204 ceph-mon[33595]: pgmap v721: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:48:12.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:11 smithi181 ceph-mon[29801]: pgmap v721: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:48:14.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:13 smithi196 ceph-mon[32858]: pgmap v722: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:48:14.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:13 smithi204 ceph-mon[33595]: pgmap v722: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:48:14.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:13 smithi181 ceph-mon[29801]: pgmap v722: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:48:16.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:15 smithi196 ceph-mon[32858]: pgmap v723: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:48:16.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:15 smithi204 ceph-mon[33595]: pgmap v723: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:48:16.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:15 smithi181 ceph-mon[29801]: pgmap v723: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:48:18.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:17 smithi196 ceph-mon[32858]: pgmap v724: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:48:18.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:17 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:48:18.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:17 smithi204 ceph-mon[33595]: pgmap v724: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:48:18.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:17 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:48:18.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:17 smithi181 ceph-mon[29801]: pgmap v724: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:48:18.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:17 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:48:19.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:18 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:48:19.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:18 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:48:19.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:18 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:48:19.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:18 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:48:19.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:18 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:48:19.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:18 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:48:19.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:18 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:48:19.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:18 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:48:19.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:18 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:48:19.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:18 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:48:19.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:18 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:48:19.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:18 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:48:19.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:18 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:48:19.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:18 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:48:19.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:18 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:48:20.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:19 smithi181 ceph-mon[29801]: pgmap v725: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:48:20.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:19 smithi196 ceph-mon[32858]: pgmap v725: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:48:20.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:19 smithi204 ceph-mon[33595]: pgmap v725: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:48:22.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:21 smithi181 ceph-mon[29801]: pgmap v726: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:48:22.355 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:21 smithi196 ceph-mon[32858]: pgmap v726: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:48:22.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:21 smithi204 ceph-mon[33595]: pgmap v726: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:48:24.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:23 smithi181 ceph-mon[29801]: pgmap v727: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:48:24.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:23 smithi196 ceph-mon[32858]: pgmap v727: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:48:24.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:23 smithi204 ceph-mon[33595]: pgmap v727: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:48:26.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:25 smithi181 ceph-mon[29801]: pgmap v728: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:48:26.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:25 smithi196 ceph-mon[32858]: pgmap v728: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:48:26.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:25 smithi204 ceph-mon[33595]: pgmap v728: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:48:28.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:27 smithi181 ceph-mon[29801]: pgmap v729: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:48:28.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:27 smithi196 ceph-mon[32858]: pgmap v729: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:48:28.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:27 smithi204 ceph-mon[33595]: pgmap v729: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:48:30.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:29 smithi181 ceph-mon[29801]: pgmap v730: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:48:30.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:29 smithi196 ceph-mon[32858]: pgmap v730: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:48:30.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:29 smithi204 ceph-mon[33595]: pgmap v730: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:48:31.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:31 smithi196 ceph-mon[32858]: pgmap v731: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:48:31.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:31 smithi204 ceph-mon[33595]: pgmap v731: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:48:31.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:31 smithi181 ceph-mon[29801]: pgmap v731: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:48:33.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:33 smithi181 ceph-mon[29801]: pgmap v732: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:48:33.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:33 smithi196 ceph-mon[32858]: pgmap v732: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:48:33.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:33 smithi204 ceph-mon[33595]: pgmap v732: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:48:35.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:35 smithi181 ceph-mon[29801]: pgmap v733: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:48:35.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:35 smithi196 ceph-mon[32858]: pgmap v733: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:48:35.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:35 smithi204 ceph-mon[33595]: pgmap v733: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:48:37.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:37 smithi181 ceph-mon[29801]: pgmap v734: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:48:37.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:37 smithi196 ceph-mon[32858]: pgmap v734: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:48:37.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:37 smithi204 ceph-mon[33595]: pgmap v734: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:48:39.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:39 smithi181 ceph-mon[29801]: pgmap v735: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:48:39.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:39 smithi196 ceph-mon[32858]: pgmap v735: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:48:39.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:39 smithi204 ceph-mon[33595]: pgmap v735: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:48:41.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:41 smithi181 ceph-mon[29801]: pgmap v736: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:48:41.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:41 smithi196 ceph-mon[32858]: pgmap v736: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:48:41.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:41 smithi204 ceph-mon[33595]: pgmap v736: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:48:43.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:43 smithi181 ceph-mon[29801]: pgmap v737: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:48:43.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:43 smithi196 ceph-mon[32858]: pgmap v737: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:48:43.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:43 smithi204 ceph-mon[33595]: pgmap v737: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:48:45.463 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:45 smithi181 ceph-mon[29801]: pgmap v738: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:48:45.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:45 smithi196 ceph-mon[32858]: pgmap v738: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:48:45.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:45 smithi204 ceph-mon[33595]: pgmap v738: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:48:47.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:47 smithi181 ceph-mon[29801]: pgmap v739: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:48:47.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:47 smithi196 ceph-mon[32858]: pgmap v739: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:48:47.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:47 smithi204 ceph-mon[33595]: pgmap v739: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:48:49.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:49 smithi181 ceph-mon[29801]: pgmap v740: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T20:48:49.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:49 smithi196 ceph-mon[32858]: pgmap v740: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T20:48:49.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:49 smithi204 ceph-mon[33595]: pgmap v740: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T20:48:51.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:51 smithi181 ceph-mon[29801]: pgmap v741: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-05-03T20:48:51.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:51 smithi196 ceph-mon[32858]: pgmap v741: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-05-03T20:48:51.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:51 smithi204 ceph-mon[33595]: pgmap v741: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-05-03T20:48:51.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:48:51.657+0000 7f420f7fe640 1 -- 172.21.15.181:0/3015235437 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f4208006100 con 0x7f4228102b20 2024-05-03T20:48:51.657 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:48:51.657+0000 7f422c81b640 1 -- 172.21.15.181:0/3015235437 <== mon.0 v2:172.21.15.181:3300/0 11 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f421c0dc070 con 0x7f4228102b20 2024-05-03T20:48:53.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:53 smithi181 ceph-mon[29801]: pgmap v742: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-05-03T20:48:53.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:53 smithi196 ceph-mon[32858]: pgmap v742: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-05-03T20:48:53.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:53 smithi204 ceph-mon[33595]: pgmap v742: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-05-03T20:48:55.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:55 smithi181 ceph-mon[29801]: pgmap v743: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 op/s 2024-05-03T20:48:55.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:55 smithi196 ceph-mon[32858]: pgmap v743: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 op/s 2024-05-03T20:48:55.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:55 smithi204 ceph-mon[33595]: pgmap v743: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 op/s 2024-05-03T20:48:57.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:57 smithi181 ceph-mon[29801]: pgmap v744: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-03T20:48:57.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:57 smithi196 ceph-mon[32858]: pgmap v744: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-03T20:48:57.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:57 smithi204 ceph-mon[33595]: pgmap v744: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-03T20:48:59.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:48:59 smithi181 ceph-mon[29801]: pgmap v745: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T20:48:59.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:48:59 smithi196 ceph-mon[32858]: pgmap v745: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T20:48:59.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:48:59 smithi204 ceph-mon[33595]: pgmap v745: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T20:49:01.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:01 smithi181 ceph-mon[29801]: pgmap v746: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:49:01.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:01 smithi196 ceph-mon[32858]: pgmap v746: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:49:01.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:01 smithi204 ceph-mon[33595]: pgmap v746: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:49:03.462 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:03 smithi181 ceph-mon[29801]: pgmap v747: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:49:03.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:03 smithi196 ceph-mon[32858]: pgmap v747: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:49:03.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:03 smithi204 ceph-mon[33595]: pgmap v747: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:49:05.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:05 smithi196 ceph-mon[32858]: pgmap v748: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:49:05.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:05 smithi204 ceph-mon[33595]: pgmap v748: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:49:05.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:05 smithi181 ceph-mon[29801]: pgmap v748: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:49:07.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:07 smithi196 ceph-mon[32858]: pgmap v749: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:49:07.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:07 smithi204 ceph-mon[33595]: pgmap v749: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:49:07.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:07 smithi181 ceph-mon[29801]: pgmap v749: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:49:09.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:09 smithi196 ceph-mon[32858]: pgmap v750: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:49:09.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:09 smithi204 ceph-mon[33595]: pgmap v750: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:49:09.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:09 smithi181 ceph-mon[29801]: pgmap v750: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:49:11.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:11 smithi196 ceph-mon[32858]: pgmap v751: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:49:11.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:11 smithi204 ceph-mon[33595]: pgmap v751: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:49:11.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:11 smithi181 ceph-mon[29801]: pgmap v751: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:49:13.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:13 smithi196 ceph-mon[32858]: pgmap v752: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:49:13.613 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:13 smithi204 ceph-mon[33595]: pgmap v752: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:49:13.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:13 smithi181 ceph-mon[29801]: pgmap v752: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:49:15.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:15 smithi196 ceph-mon[32858]: pgmap v753: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:49:15.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:15 smithi204 ceph-mon[33595]: pgmap v753: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:49:15.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:15 smithi181 ceph-mon[29801]: pgmap v753: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:49:17.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:17 smithi196 ceph-mon[32858]: pgmap v754: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:49:17.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:17 smithi204 ceph-mon[33595]: pgmap v754: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:49:17.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:17 smithi181 ceph-mon[29801]: pgmap v754: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:49:19.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:19 smithi196 ceph-mon[32858]: pgmap v755: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:49:19.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:19 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:49:19.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:19 smithi204 ceph-mon[33595]: pgmap v755: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:49:19.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:49:19.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:19 smithi181 ceph-mon[29801]: pgmap v755: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:49:19.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:19 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:49:20.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:20 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:49:20.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:20 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:49:20.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:20 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:49:20.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:20 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:49:20.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:20 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:49:20.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:20 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:49:20.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:20 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:49:20.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:20 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:49:20.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:20 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:49:21.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:21 smithi181 ceph-mon[29801]: pgmap v756: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:49:21.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:21 smithi196 ceph-mon[32858]: pgmap v756: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:49:21.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:21 smithi204 ceph-mon[33595]: pgmap v756: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:49:23.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:23 smithi181 ceph-mon[29801]: pgmap v757: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:49:23.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:23 smithi196 ceph-mon[32858]: pgmap v757: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:49:23.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:23 smithi204 ceph-mon[33595]: pgmap v757: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:49:25.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:25 smithi181 ceph-mon[29801]: pgmap v758: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:49:25.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:25 smithi196 ceph-mon[32858]: pgmap v758: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:49:25.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:25 smithi204 ceph-mon[33595]: pgmap v758: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:49:27.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:27 smithi181 ceph-mon[29801]: pgmap v759: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:49:27.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:27 smithi196 ceph-mon[32858]: pgmap v759: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:49:27.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:27 smithi204 ceph-mon[33595]: pgmap v759: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:49:29.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:29 smithi181 ceph-mon[29801]: pgmap v760: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:49:29.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:29 smithi196 ceph-mon[32858]: pgmap v760: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:49:29.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:29 smithi204 ceph-mon[33595]: pgmap v760: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:49:31.715 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:31 smithi181 ceph-mon[29801]: pgmap v761: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:49:31.855 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:31 smithi196 ceph-mon[32858]: pgmap v761: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:49:31.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:31 smithi204 ceph-mon[33595]: pgmap v761: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:49:33.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:33 smithi181 ceph-mon[29801]: pgmap v762: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:49:33.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:33 smithi196 ceph-mon[32858]: pgmap v762: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:49:33.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:33 smithi204 ceph-mon[33595]: pgmap v762: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:49:35.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:35 smithi181 ceph-mon[29801]: pgmap v763: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:49:35.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:35 smithi196 ceph-mon[32858]: pgmap v763: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:49:35.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:35 smithi204 ceph-mon[33595]: pgmap v763: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:49:37.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:37 smithi181 ceph-mon[29801]: pgmap v764: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:49:37.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:37 smithi196 ceph-mon[32858]: pgmap v764: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:49:37.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:37 smithi204 ceph-mon[33595]: pgmap v764: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:49:39.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:39 smithi181 ceph-mon[29801]: pgmap v765: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:49:39.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:39 smithi196 ceph-mon[32858]: pgmap v765: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:49:39.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:39 smithi204 ceph-mon[33595]: pgmap v765: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:49:41.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:41 smithi181 ceph-mon[29801]: pgmap v766: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:49:42.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:41 smithi196 ceph-mon[32858]: pgmap v766: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:49:42.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:41 smithi204 ceph-mon[33595]: pgmap v766: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:49:43.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:43 smithi181 ceph-mon[29801]: pgmap v767: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:49:44.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:43 smithi196 ceph-mon[32858]: pgmap v767: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:49:44.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:43 smithi204 ceph-mon[33595]: pgmap v767: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:49:45.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:45 smithi181 ceph-mon[29801]: pgmap v768: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:49:46.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:45 smithi196 ceph-mon[32858]: pgmap v768: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:49:46.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:45 smithi204 ceph-mon[33595]: pgmap v768: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:49:47.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:47 smithi181 ceph-mon[29801]: pgmap v769: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:49:48.105 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:47 smithi196 ceph-mon[32858]: pgmap v769: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:49:48.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:47 smithi204 ceph-mon[33595]: pgmap v769: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:49:49.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:49 smithi181 ceph-mon[29801]: pgmap v770: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:49:50.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:49 smithi196 ceph-mon[32858]: pgmap v770: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:49:50.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:49 smithi204 ceph-mon[33595]: pgmap v770: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:49:51.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:51 smithi181 ceph-mon[29801]: pgmap v771: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:49:52.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:51 smithi196 ceph-mon[32858]: pgmap v771: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:49:52.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:51 smithi204 ceph-mon[33595]: pgmap v771: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:49:53.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:53 smithi181 ceph-mon[29801]: pgmap v772: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:49:54.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:53 smithi196 ceph-mon[32858]: pgmap v772: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:49:54.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:53 smithi204 ceph-mon[33595]: pgmap v772: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:49:55.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:55 smithi181 ceph-mon[29801]: pgmap v773: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:49:56.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:55 smithi196 ceph-mon[32858]: pgmap v773: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:49:56.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:55 smithi204 ceph-mon[33595]: pgmap v773: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:49:57.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:57 smithi181 ceph-mon[29801]: pgmap v774: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:49:58.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:57 smithi196 ceph-mon[32858]: pgmap v774: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:49:58.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:57 smithi204 ceph-mon[33595]: pgmap v774: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:49:59.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:49:59 smithi181 ceph-mon[29801]: pgmap v775: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:50:00.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:49:59 smithi196 ceph-mon[32858]: pgmap v775: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:50:00.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:49:59 smithi204 ceph-mon[33595]: pgmap v775: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:50:00.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:00 smithi181 ceph-mon[29801]: overall HEALTH_OK 2024-05-03T20:50:01.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:00 smithi196 ceph-mon[32858]: overall HEALTH_OK 2024-05-03T20:50:01.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:00 smithi204 ceph-mon[33595]: overall HEALTH_OK 2024-05-03T20:50:01.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:01 smithi181 ceph-mon[29801]: pgmap v776: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:50:02.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:01 smithi196 ceph-mon[32858]: pgmap v776: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:50:02.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:01 smithi204 ceph-mon[33595]: pgmap v776: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:50:03.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:03 smithi181 ceph-mon[29801]: pgmap v777: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:50:04.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:03 smithi196 ceph-mon[32858]: pgmap v777: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:50:04.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:03 smithi204 ceph-mon[33595]: pgmap v777: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:50:05.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:05 smithi181 ceph-mon[29801]: pgmap v778: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:50:06.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:05 smithi196 ceph-mon[32858]: pgmap v778: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:50:06.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:05 smithi204 ceph-mon[33595]: pgmap v778: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:50:07.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:07 smithi181 ceph-mon[29801]: pgmap v779: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:50:08.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:07 smithi196 ceph-mon[32858]: pgmap v779: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:50:08.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:07 smithi204 ceph-mon[33595]: pgmap v779: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:50:09.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:09 smithi181 ceph-mon[29801]: pgmap v780: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:50:10.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:09 smithi196 ceph-mon[32858]: pgmap v780: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:50:10.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:09 smithi204 ceph-mon[33595]: pgmap v780: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:50:11.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:11 smithi181 ceph-mon[29801]: pgmap v781: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:50:12.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:11 smithi196 ceph-mon[32858]: pgmap v781: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:50:12.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:11 smithi204 ceph-mon[33595]: pgmap v781: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:50:14.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:13 smithi196 ceph-mon[32858]: pgmap v782: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:50:14.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:13 smithi204 ceph-mon[33595]: pgmap v782: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:50:14.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:13 smithi181 ceph-mon[29801]: pgmap v782: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:50:16.105 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:15 smithi196 ceph-mon[32858]: pgmap v783: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:50:16.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:15 smithi204 ceph-mon[33595]: pgmap v783: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:50:16.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:15 smithi181 ceph-mon[29801]: pgmap v783: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:50:18.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:17 smithi196 ceph-mon[32858]: pgmap v784: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:50:18.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:17 smithi204 ceph-mon[33595]: pgmap v784: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:50:18.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:17 smithi181 ceph-mon[29801]: pgmap v784: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:50:20.105 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:19 smithi196 ceph-mon[32858]: pgmap v785: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:50:20.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:19 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:50:20.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:19 smithi204 ceph-mon[33595]: pgmap v785: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:50:20.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:19 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:50:20.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:19 smithi181 ceph-mon[29801]: pgmap v785: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:50:20.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:19 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:50:21.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:20 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:50:21.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:20 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:50:21.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:20 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:50:21.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:20 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:50:21.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:20 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:50:21.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:20 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:50:21.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:20 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:50:21.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:20 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:50:21.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:20 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:50:22.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:21 smithi196 ceph-mon[32858]: pgmap v786: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:50:22.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:21 smithi204 ceph-mon[33595]: pgmap v786: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:50:22.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:21 smithi181 ceph-mon[29801]: pgmap v786: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:50:24.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:23 smithi196 ceph-mon[32858]: pgmap v787: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:50:24.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:23 smithi204 ceph-mon[33595]: pgmap v787: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:50:24.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:23 smithi181 ceph-mon[29801]: pgmap v787: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:50:26.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:25 smithi196 ceph-mon[32858]: pgmap v788: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:50:26.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:25 smithi204 ceph-mon[33595]: pgmap v788: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:50:26.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:25 smithi181 ceph-mon[29801]: pgmap v788: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:50:28.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:27 smithi196 ceph-mon[32858]: pgmap v789: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:50:28.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:27 smithi204 ceph-mon[33595]: pgmap v789: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:50:28.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:27 smithi181 ceph-mon[29801]: pgmap v789: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:50:30.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:29 smithi196 ceph-mon[32858]: pgmap v790: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:50:30.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:29 smithi204 ceph-mon[33595]: pgmap v790: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:50:30.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:29 smithi181 ceph-mon[29801]: pgmap v790: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:50:32.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:31 smithi196 ceph-mon[32858]: pgmap v791: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:50:32.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:31 smithi204 ceph-mon[33595]: pgmap v791: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:50:32.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:31 smithi181 ceph-mon[29801]: pgmap v791: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:50:34.014 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:33 smithi181 ceph-mon[29801]: pgmap v792: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-03T20:50:34.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:33 smithi196 ceph-mon[32858]: pgmap v792: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-03T20:50:34.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:33 smithi204 ceph-mon[33595]: pgmap v792: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-03T20:50:36.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:35 smithi196 ceph-mon[32858]: pgmap v793: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-05-03T20:50:36.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:35 smithi204 ceph-mon[33595]: pgmap v793: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-05-03T20:50:36.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:35 smithi181 ceph-mon[29801]: pgmap v793: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-05-03T20:50:38.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:37 smithi196 ceph-mon[32858]: pgmap v794: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T20:50:38.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:37 smithi204 ceph-mon[33595]: pgmap v794: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T20:50:38.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:37 smithi181 ceph-mon[29801]: pgmap v794: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-03T20:50:40.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:39 smithi196 ceph-mon[32858]: pgmap v795: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T20:50:40.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:39 smithi204 ceph-mon[33595]: pgmap v795: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T20:50:40.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:39 smithi181 ceph-mon[29801]: pgmap v795: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-03T20:50:42.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:41 smithi196 ceph-mon[32858]: pgmap v796: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-03T20:50:42.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:41 smithi204 ceph-mon[33595]: pgmap v796: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-03T20:50:42.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:41 smithi181 ceph-mon[29801]: pgmap v796: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-03T20:50:44.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:43 smithi196 ceph-mon[32858]: pgmap v797: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-03T20:50:44.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:43 smithi204 ceph-mon[33595]: pgmap v797: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-03T20:50:44.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:43 smithi181 ceph-mon[29801]: pgmap v797: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-03T20:50:46.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:45 smithi196 ceph-mon[32858]: pgmap v798: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:50:46.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:45 smithi204 ceph-mon[33595]: pgmap v798: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:50:46.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:45 smithi181 ceph-mon[29801]: pgmap v798: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:50:48.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:47 smithi196 ceph-mon[32858]: pgmap v799: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:50:48.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:47 smithi204 ceph-mon[33595]: pgmap v799: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:50:48.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:47 smithi181 ceph-mon[29801]: pgmap v799: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:50:50.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:49 smithi196 ceph-mon[32858]: pgmap v800: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:50:50.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:49 smithi204 ceph-mon[33595]: pgmap v800: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:50:50.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:49 smithi181 ceph-mon[29801]: pgmap v800: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:50:52.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:51 smithi196 ceph-mon[32858]: pgmap v801: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:50:52.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:51 smithi204 ceph-mon[33595]: pgmap v801: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:50:52.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:51 smithi181 ceph-mon[29801]: pgmap v801: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:50:54.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:53 smithi196 ceph-mon[32858]: pgmap v802: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:50:54.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:53 smithi204 ceph-mon[33595]: pgmap v802: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:50:54.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:53 smithi181 ceph-mon[29801]: pgmap v802: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:50:56.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:55 smithi196 ceph-mon[32858]: pgmap v803: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:50:56.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:55 smithi204 ceph-mon[33595]: pgmap v803: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:50:56.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:55 smithi181 ceph-mon[29801]: pgmap v803: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:50:58.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:57 smithi196 ceph-mon[32858]: pgmap v804: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:50:58.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:57 smithi204 ceph-mon[33595]: pgmap v804: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:50:58.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:57 smithi181 ceph-mon[29801]: pgmap v804: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:51:00.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:50:59 smithi196 ceph-mon[32858]: pgmap v805: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:51:00.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:50:59 smithi204 ceph-mon[33595]: pgmap v805: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:51:00.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:50:59 smithi181 ceph-mon[29801]: pgmap v805: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:51:02.105 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:01 smithi196 ceph-mon[32858]: pgmap v806: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:51:02.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:01 smithi204 ceph-mon[33595]: pgmap v806: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:51:02.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:01 smithi181 ceph-mon[29801]: pgmap v806: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:51:04.105 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:03 smithi196 ceph-mon[32858]: pgmap v807: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:51:04.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:03 smithi204 ceph-mon[33595]: pgmap v807: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:51:04.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:03 smithi181 ceph-mon[29801]: pgmap v807: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:51:06.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:05 smithi196 ceph-mon[32858]: pgmap v808: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:51:06.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:05 smithi204 ceph-mon[33595]: pgmap v808: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:51:06.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:05 smithi181 ceph-mon[29801]: pgmap v808: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:51:08.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:07 smithi196 ceph-mon[32858]: pgmap v809: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:51:08.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:07 smithi204 ceph-mon[33595]: pgmap v809: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:51:08.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:07 smithi181 ceph-mon[29801]: pgmap v809: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:51:10.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:09 smithi196 ceph-mon[32858]: pgmap v810: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:51:10.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:09 smithi204 ceph-mon[33595]: pgmap v810: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:51:10.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:09 smithi181 ceph-mon[29801]: pgmap v810: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:51:12.105 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:11 smithi196 ceph-mon[32858]: pgmap v811: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:51:12.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:11 smithi204 ceph-mon[33595]: pgmap v811: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:51:12.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:11 smithi181 ceph-mon[29801]: pgmap v811: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:51:14.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:13 smithi196 ceph-mon[32858]: pgmap v812: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:51:14.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:13 smithi204 ceph-mon[33595]: pgmap v812: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:51:14.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:13 smithi181 ceph-mon[29801]: pgmap v812: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:51:16.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:15 smithi196 ceph-mon[32858]: pgmap v813: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:51:16.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:15 smithi204 ceph-mon[33595]: pgmap v813: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:51:16.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:15 smithi181 ceph-mon[29801]: pgmap v813: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:51:18.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:17 smithi196 ceph-mon[32858]: pgmap v814: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:51:18.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:17 smithi204 ceph-mon[33595]: pgmap v814: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:51:18.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:17 smithi181 ceph-mon[29801]: pgmap v814: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:51:20.105 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:19 smithi196 ceph-mon[32858]: pgmap v815: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:51:20.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:19 smithi204 ceph-mon[33595]: pgmap v815: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:51:20.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:19 smithi181 ceph-mon[29801]: pgmap v815: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:51:21.105 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:20 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:51:21.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:20 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:51:21.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:20 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T20:51:22.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:21 smithi196 ceph-mon[32858]: pgmap v816: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:51:22.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:21 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:51:22.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:21 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:51:22.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:21 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:51:22.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:21 smithi204 ceph-mon[33595]: pgmap v816: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:51:22.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:21 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:51:22.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:21 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:51:22.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:21 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:51:22.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:21 smithi181 ceph-mon[29801]: pgmap v816: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:51:22.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:21 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T20:51:22.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:21 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T20:51:22.213 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:21 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' 2024-05-03T20:51:24.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:23 smithi196 ceph-mon[32858]: pgmap v817: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:51:24.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:23 smithi204 ceph-mon[33595]: pgmap v817: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:51:24.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:23 smithi181 ceph-mon[29801]: pgmap v817: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:51:26.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:25 smithi196 ceph-mon[32858]: pgmap v818: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:51:26.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:25 smithi204 ceph-mon[33595]: pgmap v818: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:51:26.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:25 smithi181 ceph-mon[29801]: pgmap v818: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:51:28.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:27 smithi196 ceph-mon[32858]: pgmap v819: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:51:28.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:27 smithi204 ceph-mon[33595]: pgmap v819: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:51:28.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:27 smithi181 ceph-mon[29801]: pgmap v819: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:51:30.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:29 smithi196 ceph-mon[32858]: pgmap v820: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:51:30.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:29 smithi204 ceph-mon[33595]: pgmap v820: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:51:30.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:29 smithi181 ceph-mon[29801]: pgmap v820: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:51:32.105 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:31 smithi196 ceph-mon[32858]: pgmap v821: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:51:32.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:31 smithi204 ceph-mon[33595]: pgmap v821: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:51:32.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:31 smithi181 ceph-mon[29801]: pgmap v821: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:51:34.105 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:33 smithi196 ceph-mon[32858]: pgmap v822: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:51:34.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:33 smithi204 ceph-mon[33595]: pgmap v822: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:51:34.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:33 smithi181 ceph-mon[29801]: pgmap v822: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:51:36.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:35 smithi196 ceph-mon[32858]: pgmap v823: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:51:36.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:35 smithi204 ceph-mon[33595]: pgmap v823: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:51:36.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:35 smithi181 ceph-mon[29801]: pgmap v823: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:51:38.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:37 smithi196 ceph-mon[32858]: pgmap v824: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:51:38.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:37 smithi204 ceph-mon[33595]: pgmap v824: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:51:38.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:37 smithi181 ceph-mon[29801]: pgmap v824: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:51:40.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:39 smithi204 ceph-mon[33595]: pgmap v825: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:51:40.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:39 smithi181 ceph-mon[29801]: pgmap v825: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:51:40.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:39 smithi196 ceph-mon[32858]: pgmap v825: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:51:42.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:41 smithi204 ceph-mon[33595]: pgmap v826: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:51:42.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:41 smithi181 ceph-mon[29801]: pgmap v826: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:51:42.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:41 smithi196 ceph-mon[32858]: pgmap v826: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:51:44.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:43 smithi181 ceph-mon[29801]: pgmap v827: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:51:44.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:43 smithi196 ceph-mon[32858]: pgmap v827: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:51:44.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:43 smithi204 ceph-mon[33595]: pgmap v827: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:51:46.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:45 smithi181 ceph-mon[29801]: pgmap v828: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:51:46.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:45 smithi196 ceph-mon[32858]: pgmap v828: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:51:46.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:45 smithi204 ceph-mon[33595]: pgmap v828: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:51:48.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:47 smithi204 ceph-mon[33595]: pgmap v829: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:51:48.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:47 smithi181 ceph-mon[29801]: pgmap v829: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:51:48.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:47 smithi196 ceph-mon[32858]: pgmap v829: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-03T20:51:50.157 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:49 smithi196 ceph-mon[32858]: pgmap v830: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:51:50.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:49 smithi181 ceph-mon[29801]: pgmap v830: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:51:50.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:49 smithi204 ceph-mon[33595]: pgmap v830: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-03T20:51:51.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:51.591+0000 7f4226ffd640 1 -- 172.21.15.181:0/3015235437 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f42100b2920 msgr2=0x7f42100b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).tick idle (900027173) for more than 900000000 us, fault. 2024-05-03T20:51:51.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:51.591+0000 7f4226ffd640 1 --2- 172.21.15.181:0/3015235437 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f42100b2920 0x7f42100b4de0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4218005db0 tx=0x7f4218005d40 comp rx=0 tx=0).stop 2024-05-03T20:51:51.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:51.591+0000 7f422c81b640 1 -- 172.21.15.181:0/3015235437 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f42100b2920 msgr2=0x7f42100b4de0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-03T20:51:51.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:51.591+0000 7f422c81b640 1 --2- 172.21.15.181:0/3015235437 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f42100b2920 0x7f42100b4de0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:51:51.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:51.591+0000 7f422c81b640 1 --2- 172.21.15.181:0/3015235437 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f42100bc930 0x7f42100bed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:51:51.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:51.591+0000 7f422c81b640 1 -- 172.21.15.181:0/3015235437 --> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f421c046e50 con 0x7f42100bc930 2024-05-03T20:51:51.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:51.592+0000 7f4226ffd640 1 --2- 172.21.15.181:0/3015235437 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f42100bc930 0x7f42100bed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:51:51.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:51.592+0000 7f4226ffd640 1 --2- 172.21.15.181:0/3015235437 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f42100bc930 0x7f42100bed20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f421800aac0 tx=0x7f4218013000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:51:51.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:51.683+0000 7f420f7fe640 1 -- 172.21.15.181:0/3015235437 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f4208006100 con 0x7f4228102b20 2024-05-03T20:51:51.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:51.684+0000 7f422c81b640 1 -- 172.21.15.181:0/3015235437 <== mon.0 v2:172.21.15.181:3300/0 12 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f4208006100 con 0x7f4228102b20 2024-05-03T20:51:51.775 INFO:teuthology.orchestra.run.smithi181.stderr:Error ETIMEDOUT: Connection timed out. Please check the client's permission and connection. 2024-05-03T20:51:51.782 DEBUG:teuthology.orchestra.run:got remote process result: 110 2024-05-03T20:51:51.783 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/exec.py", line 66, in task remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi181 with status 110: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10'" 2024-05-03T20:51:52.034 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=06ebc29797f645da8539fafcabe31f74 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/exec.py", line 66, in task remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi181 with status 110: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10'" 2024-05-03T20:51:52.036 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-03T20:51:52.049 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-03T20:51:52.049 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-03T20:51:52.049 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-03T20:51:52.073 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-03T20:51:52.073 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-03T20:51:52.131 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-03T20:51:52.195 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:51 smithi181 ceph-mon[29801]: pgmap v831: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:51:52.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:51 smithi196 ceph-mon[32858]: pgmap v831: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:51:52.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:51 smithi204 ceph-mon[33595]: pgmap v831: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-03T20:51:52.417 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:51:53.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:52 smithi181 ceph-mon[29801]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:51:53.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:52 smithi196 ceph-mon[32858]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:51:53.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:52 smithi204 ceph-mon[33595]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T20:51:54.117 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.117+0000 7f85e42ff700 1 -- 172.21.15.181:0/3996602976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85dc075ff0 msgr2=0x7f85dc10f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:51:54.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.117+0000 7f85e42ff700 1 --2- 172.21.15.181:0/3996602976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85dc075ff0 0x7f85dc10f960 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009990 tx=0x7f85d0009ca0 comp rx=0 tx=0).stop 2024-05-03T20:51:54.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.117+0000 7f85e42ff700 1 -- 172.21.15.181:0/3996602976 shutdown_connections 2024-05-03T20:51:54.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.117+0000 7f85e42ff700 1 --2- 172.21.15.181:0/3996602976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85dc10fea0 0x7f85dc112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:51:54.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.117+0000 7f85e42ff700 1 --2- 172.21.15.181:0/3996602976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85dc0756d0 0x7f85dc075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:51:54.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.117+0000 7f85e42ff700 1 --2- 172.21.15.181:0/3996602976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85dc075ff0 0x7f85dc10f960 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:51:54.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.117+0000 7f85e42ff700 1 -- 172.21.15.181:0/3996602976 >> 172.21.15.181:0/3996602976 conn(0x7f85dc07a650 msgr2=0x7f85dc07aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:51:54.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.117+0000 7f85e42ff700 1 -- 172.21.15.181:0/3996602976 shutdown_connections 2024-05-03T20:51:54.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.117+0000 7f85e42ff700 1 -- 172.21.15.181:0/3996602976 wait complete. 2024-05-03T20:51:54.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.118+0000 7f85e42ff700 1 Processor -- start 2024-05-03T20:51:54.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.118+0000 7f85e42ff700 1 -- start start 2024-05-03T20:51:54.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.118+0000 7f85e42ff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85dc0756d0 0x7f85dc19be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:51:54.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.118+0000 7f85e42ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85dc075ff0 0x7f85dc19c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:51:54.120 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.118+0000 7f85e42ff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85dc10fea0 0x7f85dc1a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:51:54.120 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.118+0000 7f85e42ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc115920 con 0x7f85dc075ff0 2024-05-03T20:51:54.120 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.118+0000 7f85e42ff700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc1157a0 con 0x7f85dc10fea0 2024-05-03T20:51:54.120 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.118+0000 7f85e42ff700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc115aa0 con 0x7f85dc0756d0 2024-05-03T20:51:54.120 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.118+0000 7f85e189a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85dc075ff0 0x7f85dc19c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:51:54.120 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.118+0000 7f85e189a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85dc075ff0 0x7f85dc19c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47744/0 (socket says 172.21.15.181:47744) 2024-05-03T20:51:54.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.118+0000 7f85e189a700 1 -- 172.21.15.181:0/632726308 learned_addr learned my addr 172.21.15.181:0/632726308 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:51:54.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.118+0000 7f85e209b700 1 --2- 172.21.15.181:0/632726308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85dc0756d0 0x7f85dc19be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:51:54.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.118+0000 7f85e289c700 1 --2- 172.21.15.181:0/632726308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85dc10fea0 0x7f85dc1a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:51:54.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.118+0000 7f85e189a700 1 -- 172.21.15.181:0/632726308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85dc0756d0 msgr2=0x7f85dc19be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:51:54.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.118+0000 7f85e189a700 1 --2- 172.21.15.181:0/632726308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85dc0756d0 0x7f85dc19be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:51:54.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.118+0000 7f85e189a700 1 -- 172.21.15.181:0/632726308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85dc10fea0 msgr2=0x7f85dc1a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:51:54.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.118+0000 7f85e189a700 1 --2- 172.21.15.181:0/632726308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85dc10fea0 0x7f85dc1a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:51:54.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.118+0000 7f85e189a700 1 -- 172.21.15.181:0/632726308 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85dc1a0e30 con 0x7f85dc075ff0 2024-05-03T20:51:54.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.118+0000 7f85e209b700 1 --2- 172.21.15.181:0/632726308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85dc0756d0 0x7f85dc19be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:51:54.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.118+0000 7f85e289c700 1 --2- 172.21.15.181:0/632726308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85dc10fea0 0x7f85dc1a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:51:54.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.118+0000 7f85e189a700 1 --2- 172.21.15.181:0/632726308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85dc075ff0 0x7f85dc19c3b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f85d0005f30 tx=0x7f85d0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:51:54.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.119+0000 7f85db7fe700 1 -- 172.21.15.181:0/632726308 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d0027070 con 0x7f85dc075ff0 2024-05-03T20:51:54.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.119+0000 7f85e42ff700 1 -- 172.21.15.181:0/632726308 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85dc1a10c0 con 0x7f85dc075ff0 2024-05-03T20:51:54.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.119+0000 7f85db7fe700 1 -- 172.21.15.181:0/632726308 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85d0003fb0 con 0x7f85dc075ff0 2024-05-03T20:51:54.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.119+0000 7f85db7fe700 1 -- 172.21.15.181:0/632726308 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d0022d40 con 0x7f85dc075ff0 2024-05-03T20:51:54.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.119+0000 7f85e42ff700 1 -- 172.21.15.181:0/632726308 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85dc1a89a0 con 0x7f85dc075ff0 2024-05-03T20:51:54.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.120+0000 7f85e42ff700 1 -- 172.21.15.181:0/632726308 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85dc076df0 con 0x7f85dc075ff0 2024-05-03T20:51:54.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.121+0000 7f85db7fe700 1 -- 172.21.15.181:0/632726308 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f85d002a070 con 0x7f85dc075ff0 2024-05-03T20:51:54.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.122+0000 7f85db7fe700 1 --2- 172.21.15.181:0/632726308 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f85c40b2910 0x7f85c40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:51:54.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.122+0000 7f85e209b700 1 --2- 172.21.15.181:0/632726308 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f85c40b2910 0x7f85c40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:51:54.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.122+0000 7f85db7fe700 1 -- 172.21.15.181:0/632726308 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f85d00e3910 con 0x7f85dc075ff0 2024-05-03T20:51:54.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.122+0000 7f85e209b700 1 --2- 172.21.15.181:0/632726308 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f85c40b2910 0x7f85c40b4dd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f85cc009dd0 tx=0x7f85cc009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:51:54.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.126+0000 7f85db7fe700 1 -- 172.21.15.181:0/632726308 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f85d0031020 con 0x7f85dc075ff0 2024-05-03T20:51:54.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:53 smithi181 ceph-mon[29801]: pgmap v832: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:51:54.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.355+0000 7f85e42ff700 1 -- 172.21.15.181:0/632726308 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f85dc19d180 con 0x7f85dc075ff0 2024-05-03T20:51:54.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.356+0000 7f85db7fe700 1 -- 172.21.15.181:0/632726308 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v83) ==== 81+0+599 (secure 0 0 0) 0x7f85d0013070 con 0x7f85dc075ff0 2024-05-03T20:51:54.355 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6800/2389231557 2024-05-04T20:26:33.766772+0000 2024-05-03T20:51:54.355 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/3398276284 2024-05-04T20:26:06.561713+0000 2024-05-03T20:51:54.355 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6801/2389231557 2024-05-04T20:26:33.766772+0000 2024-05-03T20:51:54.355 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/2327733609 2024-05-04T20:26:06.561713+0000 2024-05-03T20:51:54.356 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/628540759 2024-05-04T20:26:33.766772+0000 2024-05-03T20:51:54.356 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/3387163329 2024-05-04T20:26:06.561713+0000 2024-05-03T20:51:54.356 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6800/1802984663 2024-05-04T20:26:06.561713+0000 2024-05-03T20:51:54.356 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/67924109 2024-05-04T20:26:33.766772+0000 2024-05-03T20:51:54.356 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/3693389657 2024-05-04T20:26:33.766772+0000 2024-05-03T20:51:54.357 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6801/1802984663 2024-05-04T20:26:06.561713+0000 2024-05-03T20:51:54.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:53 smithi196 ceph-mon[32858]: pgmap v832: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:51:54.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.358+0000 7f85e42ff700 1 -- 172.21.15.181:0/632726308 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f85c40b2910 msgr2=0x7f85c40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:51:54.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.358+0000 7f85e42ff700 1 --2- 172.21.15.181:0/632726308 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f85c40b2910 0x7f85c40b4dd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f85cc009dd0 tx=0x7f85cc009040 comp rx=0 tx=0).stop 2024-05-03T20:51:54.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.358+0000 7f85e42ff700 1 -- 172.21.15.181:0/632726308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85dc075ff0 msgr2=0x7f85dc19c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:51:54.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.358+0000 7f85e42ff700 1 --2- 172.21.15.181:0/632726308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85dc075ff0 0x7f85dc19c3b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f85d0005f30 tx=0x7f85d0009f90 comp rx=0 tx=0).stop 2024-05-03T20:51:54.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.358+0000 7f85e42ff700 1 -- 172.21.15.181:0/632726308 shutdown_connections 2024-05-03T20:51:54.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.358+0000 7f85e42ff700 1 --2- 172.21.15.181:0/632726308 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f85c40b2910 0x7f85c40b4dd0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:51:54.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.358+0000 7f85e42ff700 1 --2- 172.21.15.181:0/632726308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85dc10fea0 0x7f85dc1a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:51:54.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.358+0000 7f85e42ff700 1 --2- 172.21.15.181:0/632726308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85dc0756d0 0x7f85dc19be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:51:54.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.358+0000 7f85e42ff700 1 --2- 172.21.15.181:0/632726308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85dc075ff0 0x7f85dc19c3b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:51:54.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.358+0000 7f85e42ff700 1 -- 172.21.15.181:0/632726308 >> 172.21.15.181:0/632726308 conn(0x7f85dc07a650 msgr2=0x7f85dc100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:51:54.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.358+0000 7f85e42ff700 1 -- 172.21.15.181:0/632726308 shutdown_connections 2024-05-03T20:51:54.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:54.358+0000 7f85e42ff700 1 -- 172.21.15.181:0/632726308 wait complete. 2024-05-03T20:51:54.360 INFO:teuthology.orchestra.run.smithi181.stderr:listed 10 entries 2024-05-03T20:51:54.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:53 smithi204 ceph-mon[33595]: pgmap v832: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-03T20:51:54.968 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-03T20:51:54.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T20:51:54.969 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-03T20:51:55.043 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2024-05-03T20:51:55.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T20:51:55.043 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-03T20:51:55.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T20:51:55.109 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-05-03T20:51:55.164 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-03T20:51:55.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T20:51:55.164 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-03T20:51:55.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:54 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/632726308' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T20:51:55.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T20:51:55.228 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link delete ceph-brx 2024-05-03T20:51:55.332 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-03T20:51:55.341 INFO:tasks.cephadm:Teardown begin 2024-05-03T20:51:55.342 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-03T20:51:55.342 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T20:51:55.355 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:54 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/632726308' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T20:51:55.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:54 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/632726308' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T20:51:55.586 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:51:55.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:55 smithi181 ceph-mon[29801]: pgmap v833: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:51:56.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:55 smithi196 ceph-mon[32858]: pgmap v833: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:51:56.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:55 smithi204 ceph-mon[33595]: pgmap v833: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-03T20:51:57.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.179+0000 7fd7a2f43700 1 -- 172.21.15.181:0/2540753325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd79c102d60 msgr2=0x7fd79c10f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:51:57.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.179+0000 7fd7a2f43700 1 --2- 172.21.15.181:0/2540753325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd79c102d60 0x7fd79c10f250 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd784009990 tx=0x7fd784009ca0 comp rx=0 tx=0).stop 2024-05-03T20:51:57.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.179+0000 7fd7a2f43700 1 -- 172.21.15.181:0/2540753325 shutdown_connections 2024-05-03T20:51:57.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.179+0000 7fd7a2f43700 1 --2- 172.21.15.181:0/2540753325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd79c102d60 0x7fd79c10f250 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:51:57.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.179+0000 7fd7a2f43700 1 --2- 172.21.15.181:0/2540753325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd79c108410 0x7fd79c1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:51:57.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.179+0000 7fd7a2f43700 1 --2- 172.21.15.181:0/2540753325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd79c1023a0 0x7fd79c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:51:57.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.179+0000 7fd7a2f43700 1 -- 172.21.15.181:0/2540753325 >> 172.21.15.181:0/2540753325 conn(0x7fd79c079350 msgr2=0x7fd79c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:51:57.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.179+0000 7fd7a2f43700 1 -- 172.21.15.181:0/2540753325 shutdown_connections 2024-05-03T20:51:57.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.179+0000 7fd7a2f43700 1 -- 172.21.15.181:0/2540753325 wait complete. 2024-05-03T20:51:57.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.180+0000 7fd7a2f43700 1 Processor -- start 2024-05-03T20:51:57.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.180+0000 7fd7a2f43700 1 -- start start 2024-05-03T20:51:57.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.180+0000 7fd7a2f43700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd79c1023a0 0x7fd79c19bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:51:57.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.180+0000 7fd7a2f43700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd79c102d60 0x7fd79c19c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:51:57.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.180+0000 7fd7a2f43700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd79c108410 0x7fd79c1a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:51:57.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.180+0000 7fd7a2f43700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c1125b0 con 0x7fd79c1023a0 2024-05-03T20:51:57.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.180+0000 7fd7a2f43700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c112430 con 0x7fd79c108410 2024-05-03T20:51:57.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.180+0000 7fd7a2f43700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c112730 con 0x7fd79c102d60 2024-05-03T20:51:57.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.180+0000 7fd7a0cdf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd79c1023a0 0x7fd79c19bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:51:57.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.180+0000 7fd7a0cdf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd79c1023a0 0x7fd79c19bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49202/0 (socket says 172.21.15.181:49202) 2024-05-03T20:51:57.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.180+0000 7fd7a0cdf700 1 -- 172.21.15.181:0/2670430985 learned_addr learned my addr 172.21.15.181:0/2670430985 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:51:57.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.180+0000 7fd79bfff700 1 --2- 172.21.15.181:0/2670430985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd79c102d60 0x7fd79c19c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:51:57.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.180+0000 7fd7a14e0700 1 --2- 172.21.15.181:0/2670430985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd79c108410 0x7fd79c1a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:51:57.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.180+0000 7fd7a0cdf700 1 -- 172.21.15.181:0/2670430985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd79c102d60 msgr2=0x7fd79c19c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:51:57.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.180+0000 7fd7a0cdf700 1 --2- 172.21.15.181:0/2670430985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd79c102d60 0x7fd79c19c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:51:57.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.180+0000 7fd7a0cdf700 1 -- 172.21.15.181:0/2670430985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd79c108410 msgr2=0x7fd79c1a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:51:57.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.180+0000 7fd7a0cdf700 1 --2- 172.21.15.181:0/2670430985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd79c108410 0x7fd79c1a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:51:57.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.180+0000 7fd7a0cdf700 1 -- 172.21.15.181:0/2670430985 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd79c1a0d40 con 0x7fd79c1023a0 2024-05-03T20:51:57.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.180+0000 7fd79bfff700 1 --2- 172.21.15.181:0/2670430985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd79c102d60 0x7fd79c19c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:51:57.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.180+0000 7fd7a14e0700 1 --2- 172.21.15.181:0/2670430985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd79c108410 0x7fd79c1a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T20:51:57.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.181+0000 7fd7a0cdf700 1 --2- 172.21.15.181:0/2670430985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd79c1023a0 0x7fd79c19bd30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd79400ba50 tx=0x7fd79400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:51:57.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.181+0000 7fd799ffb700 1 -- 172.21.15.181:0/2670430985 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd794019070 con 0x7fd79c1023a0 2024-05-03T20:51:57.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.181+0000 7fd799ffb700 1 -- 172.21.15.181:0/2670430985 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd794014440 con 0x7fd79c1023a0 2024-05-03T20:51:57.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.181+0000 7fd7a2f43700 1 -- 172.21.15.181:0/2670430985 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd79c1a1030 con 0x7fd79c1023a0 2024-05-03T20:51:57.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.181+0000 7fd799ffb700 1 -- 172.21.15.181:0/2670430985 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7940075d0 con 0x7fd79c1023a0 2024-05-03T20:51:57.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.181+0000 7fd7a2f43700 1 -- 172.21.15.181:0/2670430985 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd79c1a88b0 con 0x7fd79c1023a0 2024-05-03T20:51:57.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.182+0000 7fd7a2f43700 1 -- 172.21.15.181:0/2670430985 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd79c06a190 con 0x7fd79c1023a0 2024-05-03T20:51:57.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.184+0000 7fd799ffb700 1 -- 172.21.15.181:0/2670430985 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fd79401c070 con 0x7fd79c1023a0 2024-05-03T20:51:57.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.185+0000 7fd799ffb700 1 --2- 172.21.15.181:0/2670430985 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd7880b2910 0x7fd7880b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:51:57.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.185+0000 7fd79bfff700 1 --2- 172.21.15.181:0/2670430985 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd7880b2910 0x7fd7880b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:51:57.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.185+0000 7fd799ffb700 1 -- 172.21.15.181:0/2670430985 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7fd7940e1c60 con 0x7fd79c1023a0 2024-05-03T20:51:57.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.185+0000 7fd79bfff700 1 --2- 172.21.15.181:0/2670430985 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd7880b2910 0x7fd7880b4dd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd78c005fd0 tx=0x7fd78c005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:51:57.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.188+0000 7fd799ffb700 1 -- 172.21.15.181:0/2670430985 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd7940a9930 con 0x7fd79c1023a0 2024-05-03T20:51:57.441 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.443+0000 7fd7a2f43700 1 -- 172.21.15.181:0/2670430985 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd79c19d090 con 0x7fd79c1023a0 2024-05-03T20:51:57.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.444+0000 7fd799ffb700 1 -- 172.21.15.181:0/2670430985 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11954 (secure 0 0 0) 0x7fd7940a9230 con 0x7fd79c1023a0 2024-05-03T20:51:57.443 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:51:57.443 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2414172454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2414172454},{"type":"v1","addr":"172.21.15.181:6835","nonce":2414172454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/204127189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":204127189},{"type":"v1","addr":"172.21.15.181:6837","nonce":204127189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T20:33:42.487965+0000","modified":"2024-05-03T20:36:50.585508+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24475,"mds_2":24464,"mds_3":24463,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14562":{"gid":14562,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6839/257361276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":257361276},{"type":"v1","addr":"172.21.15.181:6839","nonce":257361276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6841/2403160115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2403160115},{"type":"v1","addr":"172.21.15.181:6841","nonce":2403160115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3975688022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3975688022},{"type":"v1","addr":"172.21.15.196:6833","nonce":3975688022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2409628823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2409628823},{"type":"v1","addr":"172.21.15.204:6833","nonce":2409628823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1820206106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1820206106},{"type":"v1","addr":"172.21.15.196:6835","nonce":1820206106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6835/3870850287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3870850287},{"type":"v1","addr":"172.21.15.204:6835","nonce":3870850287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/3006728559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3006728559},{"type":"v1","addr":"172.21.15.204:6837","nonce":3006728559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6837/1529420259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1529420259},{"type":"v1","addr":"172.21.15.196:6837","nonce":1529420259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/1644828559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1644828559},{"type":"v1","addr":"172.21.15.204:6839","nonce":1644828559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.196:6839/1182826743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1182826743},{"type":"v1","addr":"172.21.15.196:6839","nonce":1182826743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24475,24463,24457,24464]},"id":1}]} 2024-05-03T20:51:57.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.447+0000 7fd7a2f43700 1 -- 172.21.15.181:0/2670430985 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd7880b2910 msgr2=0x7fd7880b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:51:57.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.447+0000 7fd7a2f43700 1 --2- 172.21.15.181:0/2670430985 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd7880b2910 0x7fd7880b4dd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd78c005fd0 tx=0x7fd78c005dc0 comp rx=0 tx=0).stop 2024-05-03T20:51:57.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.447+0000 7fd7a2f43700 1 -- 172.21.15.181:0/2670430985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd79c1023a0 msgr2=0x7fd79c19bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:51:57.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.447+0000 7fd7a2f43700 1 --2- 172.21.15.181:0/2670430985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd79c1023a0 0x7fd79c19bd30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd79400ba50 tx=0x7fd79400be10 comp rx=0 tx=0).stop 2024-05-03T20:51:57.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.447+0000 7fd7a2f43700 1 -- 172.21.15.181:0/2670430985 shutdown_connections 2024-05-03T20:51:57.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.447+0000 7fd7a2f43700 1 --2- 172.21.15.181:0/2670430985 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fd7880b2910 0x7fd7880b4dd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:51:57.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.447+0000 7fd7a2f43700 1 --2- 172.21.15.181:0/2670430985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd79c108410 0x7fd79c1a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:51:57.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.447+0000 7fd7a2f43700 1 --2- 172.21.15.181:0/2670430985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd79c1023a0 0x7fd79c19bd30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:51:57.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.447+0000 7fd7a2f43700 1 --2- 172.21.15.181:0/2670430985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd79c102d60 0x7fd79c19c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:51:57.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.447+0000 7fd7a2f43700 1 -- 172.21.15.181:0/2670430985 >> 172.21.15.181:0/2670430985 conn(0x7fd79c079350 msgr2=0x7fd79c06b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:51:57.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.447+0000 7fd7a2f43700 1 -- 172.21.15.181:0/2670430985 shutdown_connections 2024-05-03T20:51:57.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:57.447+0000 7fd7a2f43700 1 -- 172.21.15.181:0/2670430985 wait complete. 2024-05-03T20:51:57.447 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 28 2024-05-03T20:51:58.096 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:57 smithi181 ceph-mon[29801]: pgmap v834: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-03T20:51:58.096 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:57 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/2670430985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:51:58.119 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T20:51:58.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:57 smithi196 ceph-mon[32858]: pgmap v834: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-03T20:51:58.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:57 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/2670430985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:51:58.360 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:51:58.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:57 smithi204 ceph-mon[33595]: pgmap v834: 97 pgs: 97 active+clean; 641 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-03T20:51:58.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:57 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/2670430985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:51:59.949 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.949+0000 7f46b7b27700 1 -- 172.21.15.181:0/1027435006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46b0102820 msgr2=0x7f46b0110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:51:59.950 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.949+0000 7f46b7b27700 1 --2- 172.21.15.181:0/1027435006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46b0102820 0x7f46b0110290 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f46a800b0a0 tx=0x7f46a800b3b0 comp rx=0 tx=0).stop 2024-05-03T20:51:59.950 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.949+0000 7f46b7b27700 1 -- 172.21.15.181:0/1027435006 shutdown_connections 2024-05-03T20:51:59.950 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.949+0000 7f46b7b27700 1 --2- 172.21.15.181:0/1027435006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46b0102820 0x7f46b0110290 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:51:59.950 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.949+0000 7f46b7b27700 1 --2- 172.21.15.181:0/1027435006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46b0108580 0x7f46b0108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:51:59.950 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.949+0000 7f46b7b27700 1 --2- 172.21.15.181:0/1027435006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46b0101e60 0x7f46b01022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:51:59.950 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.949+0000 7f46b7b27700 1 -- 172.21.15.181:0/1027435006 >> 172.21.15.181:0/1027435006 conn(0x7f46b00776c0 msgr2=0x7f46b0077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:51:59.950 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.949+0000 7f46b7b27700 1 -- 172.21.15.181:0/1027435006 shutdown_connections 2024-05-03T20:51:59.951 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.949+0000 7f46b7b27700 1 -- 172.21.15.181:0/1027435006 wait complete. 2024-05-03T20:51:59.951 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.950+0000 7f46b7b27700 1 Processor -- start 2024-05-03T20:51:59.951 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.950+0000 7f46b7b27700 1 -- start start 2024-05-03T20:51:59.951 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.950+0000 7f46b7b27700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46b0101e60 0x7f46b019f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:51:59.951 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.950+0000 7f46b7b27700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46b0102820 0x7f46b019fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:51:59.951 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.950+0000 7f46b7b27700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46b0108580 0x7f46b0199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:51:59.951 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.950+0000 7f46b7b27700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f46b0113790 con 0x7f46b0101e60 2024-05-03T20:51:59.951 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.950+0000 7f46b7b27700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f46b0113610 con 0x7f46b0102820 2024-05-03T20:51:59.951 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.950+0000 7f46b7b27700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f46b0113910 con 0x7f46b0108580 2024-05-03T20:51:59.952 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.950+0000 7f46b58c3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46b0101e60 0x7f46b019f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:51:59.952 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.950+0000 7f46b58c3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46b0101e60 0x7f46b019f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49212/0 (socket says 172.21.15.181:49212) 2024-05-03T20:51:59.952 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.950+0000 7f46b58c3700 1 -- 172.21.15.181:0/1022781892 learned_addr learned my addr 172.21.15.181:0/1022781892 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:51:59.952 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.950+0000 7f46b60c4700 1 --2- 172.21.15.181:0/1022781892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46b0108580 0x7f46b0199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:51:59.952 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.950+0000 7f46b50c2700 1 --2- 172.21.15.181:0/1022781892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46b0102820 0x7f46b019fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:51:59.952 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.951+0000 7f46b58c3700 1 -- 172.21.15.181:0/1022781892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46b0108580 msgr2=0x7f46b0199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:51:59.952 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.951+0000 7f46b58c3700 1 --2- 172.21.15.181:0/1022781892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46b0108580 0x7f46b0199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:51:59.952 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.951+0000 7f46b58c3700 1 -- 172.21.15.181:0/1022781892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46b0102820 msgr2=0x7f46b019fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:51:59.952 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.951+0000 7f46b58c3700 1 --2- 172.21.15.181:0/1022781892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46b0102820 0x7f46b019fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:51:59.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.951+0000 7f46b58c3700 1 -- 172.21.15.181:0/1022781892 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b019a210 con 0x7f46b0101e60 2024-05-03T20:51:59.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.951+0000 7f46b58c3700 1 --2- 172.21.15.181:0/1022781892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46b0101e60 0x7f46b019f8f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f46a000b500 tx=0x7f46a000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:51:59.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.951+0000 7f46a6ffd700 1 -- 172.21.15.181:0/1022781892 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0015020 con 0x7f46b0101e60 2024-05-03T20:51:59.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.951+0000 7f46b7b27700 1 -- 172.21.15.181:0/1022781892 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b019a500 con 0x7f46b0101e60 2024-05-03T20:51:59.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.951+0000 7f46a6ffd700 1 -- 172.21.15.181:0/1022781892 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46a0011ce0 con 0x7f46b0101e60 2024-05-03T20:51:59.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.951+0000 7f46a6ffd700 1 -- 172.21.15.181:0/1022781892 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0010d20 con 0x7f46b0101e60 2024-05-03T20:51:59.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.951+0000 7f46b7b27700 1 -- 172.21.15.181:0/1022781892 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b0102f70 con 0x7f46b0101e60 2024-05-03T20:51:59.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.952+0000 7f46b7b27700 1 -- 172.21.15.181:0/1022781892 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46b006a190 con 0x7f46b0101e60 2024-05-03T20:51:59.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.954+0000 7f46a6ffd700 1 -- 172.21.15.181:0/1022781892 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f46a0018020 con 0x7f46b0101e60 2024-05-03T20:51:59.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.955+0000 7f46a6ffd700 1 --2- 172.21.15.181:0/1022781892 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f469c0b2910 0x7f469c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:51:59.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.955+0000 7f46b50c2700 1 --2- 172.21.15.181:0/1022781892 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f469c0b2910 0x7f469c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:51:59.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.955+0000 7f46a6ffd700 1 -- 172.21.15.181:0/1022781892 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f46a00e1f60 con 0x7f46b0101e60 2024-05-03T20:51:59.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.955+0000 7f46b50c2700 1 --2- 172.21.15.181:0/1022781892 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f469c0b2910 0x7f469c0b4dd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f46ac006fd0 tx=0x7f46ac008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:51:59.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:51:59.958+0000 7f46a6ffd700 1 -- 172.21.15.181:0/1022781892 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f46a001f020 con 0x7f46b0101e60 2024-05-03T20:52:00.212 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:51:59 smithi181 ceph-mon[29801]: pgmap v835: 97 pgs: 97 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-03T20:52:00.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:00.215+0000 7f46b7b27700 1 -- 172.21.15.181:0/1022781892 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46b019afe0 con 0x7f46b0101e60 2024-05-03T20:52:00.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:00.216+0000 7f46a6ffd700 1 -- 172.21.15.181:0/1022781892 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11954 (secure 0 0 0) 0x7f46a00a9c30 con 0x7f46b0101e60 2024-05-03T20:52:00.214 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:52:00.215 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2414172454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2414172454},{"type":"v1","addr":"172.21.15.181:6835","nonce":2414172454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/204127189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":204127189},{"type":"v1","addr":"172.21.15.181:6837","nonce":204127189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T20:33:42.487965+0000","modified":"2024-05-03T20:36:50.585508+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24475,"mds_2":24464,"mds_3":24463,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14562":{"gid":14562,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6839/257361276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":257361276},{"type":"v1","addr":"172.21.15.181:6839","nonce":257361276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6841/2403160115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6840","nonce":2403160115},{"type":"v1","addr":"172.21.15.181:6841","nonce":2403160115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3975688022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3975688022},{"type":"v1","addr":"172.21.15.196:6833","nonce":3975688022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2409628823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2409628823},{"type":"v1","addr":"172.21.15.204:6833","nonce":2409628823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1820206106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1820206106},{"type":"v1","addr":"172.21.15.196:6835","nonce":1820206106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.204:6835/3870850287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3870850287},{"type":"v1","addr":"172.21.15.204:6835","nonce":3870850287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/3006728559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3006728559},{"type":"v1","addr":"172.21.15.204:6837","nonce":3006728559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6837/1529420259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1529420259},{"type":"v1","addr":"172.21.15.196:6837","nonce":1529420259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/1644828559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1644828559},{"type":"v1","addr":"172.21.15.204:6839","nonce":1644828559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.196:6839/1182826743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1182826743},{"type":"v1","addr":"172.21.15.196:6839","nonce":1182826743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24475,24463,24457,24464]},"id":1}]} 2024-05-03T20:52:00.217 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:00.219+0000 7f46b7b27700 1 -- 172.21.15.181:0/1022781892 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f469c0b2910 msgr2=0x7f469c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:52:00.217 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:00.219+0000 7f46b7b27700 1 --2- 172.21.15.181:0/1022781892 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f469c0b2910 0x7f469c0b4dd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f46ac006fd0 tx=0x7f46ac008040 comp rx=0 tx=0).stop 2024-05-03T20:52:00.218 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:00.219+0000 7f46b7b27700 1 -- 172.21.15.181:0/1022781892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46b0101e60 msgr2=0x7f46b019f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:52:00.218 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:00.219+0000 7f46b7b27700 1 --2- 172.21.15.181:0/1022781892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46b0101e60 0x7f46b019f8f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f46a000b500 tx=0x7f46a000b810 comp rx=0 tx=0).stop 2024-05-03T20:52:00.218 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:00.219+0000 7f46b7b27700 1 -- 172.21.15.181:0/1022781892 shutdown_connections 2024-05-03T20:52:00.218 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:00.219+0000 7f46b7b27700 1 --2- 172.21.15.181:0/1022781892 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f469c0b2910 0x7f469c0b4dd0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:00.218 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:00.219+0000 7f46b7b27700 1 --2- 172.21.15.181:0/1022781892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46b0108580 0x7f46b0199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:00.218 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:00.219+0000 7f46b7b27700 1 --2- 172.21.15.181:0/1022781892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46b0101e60 0x7f46b019f8f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:00.219 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:00.219+0000 7f46b7b27700 1 --2- 172.21.15.181:0/1022781892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46b0102820 0x7f46b019fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:00.219 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:00.219+0000 7f46b7b27700 1 -- 172.21.15.181:0/1022781892 >> 172.21.15.181:0/1022781892 conn(0x7f46b00776c0 msgr2=0x7f46b0100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:52:00.219 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:00.220+0000 7f46b7b27700 1 -- 172.21.15.181:0/1022781892 shutdown_connections 2024-05-03T20:52:00.219 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:00.220+0000 7f46b7b27700 1 -- 172.21.15.181:0/1022781892 wait complete. 2024-05-03T20:52:00.219 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 28 2024-05-03T20:52:00.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:51:59 smithi196 ceph-mon[32858]: pgmap v835: 97 pgs: 97 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-03T20:52:00.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:51:59 smithi204 ceph-mon[33595]: pgmap v835: 97 pgs: 97 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-03T20:52:00.819 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T20:52:01.060 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:52:01.182 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:00 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/1022781892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:52:01.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:00 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/1022781892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:52:01.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:00 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/1022781892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T20:52:02.192 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:01 smithi181 ceph-mon[29801]: pgmap v836: 97 pgs: 97 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-03T20:52:02.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:01 smithi196 ceph-mon[32858]: pgmap v836: 97 pgs: 97 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-03T20:52:02.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:01 smithi204 ceph-mon[33595]: pgmap v836: 97 pgs: 97 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-03T20:52:02.650 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.651+0000 7f197d09a700 1 -- 172.21.15.181:0/2629682167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1978109570 msgr2=0x7f1978109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:52:02.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.651+0000 7f197d09a700 1 --2- 172.21.15.181:0/2629682167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1978109570 0x7f1978109950 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f19680099c0 tx=0x7f1968009cd0 comp rx=0 tx=0).stop 2024-05-03T20:52:02.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.651+0000 7f197d09a700 1 -- 172.21.15.181:0/2629682167 shutdown_connections 2024-05-03T20:52:02.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.651+0000 7f197d09a700 1 --2- 172.21.15.181:0/2629682167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19781023b0 0x7f1978110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:02.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.651+0000 7f197d09a700 1 --2- 172.21.15.181:0/2629682167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1978109570 0x7f1978109950 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:02.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.651+0000 7f197d09a700 1 --2- 172.21.15.181:0/2629682167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19781019f0 0x7f1978101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:02.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.651+0000 7f197d09a700 1 -- 172.21.15.181:0/2629682167 >> 172.21.15.181:0/2629682167 conn(0x7f19780789a0 msgr2=0x7f19780ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:52:02.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.651+0000 7f197d09a700 1 -- 172.21.15.181:0/2629682167 shutdown_connections 2024-05-03T20:52:02.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.651+0000 7f197d09a700 1 -- 172.21.15.181:0/2629682167 wait complete. 2024-05-03T20:52:02.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.652+0000 7f197d09a700 1 Processor -- start 2024-05-03T20:52:02.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.652+0000 7f197d09a700 1 -- start start 2024-05-03T20:52:02.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.652+0000 7f197d09a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19781019f0 0x7f197819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:52:02.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.652+0000 7f197d09a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19781023b0 0x7f197819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:52:02.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.652+0000 7f197d09a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1978109570 0x7f1978199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:52:02.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.652+0000 7f197d09a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1978113650 con 0x7f19781023b0 2024-05-03T20:52:02.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.652+0000 7f197d09a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f19781134d0 con 0x7f1978109570 2024-05-03T20:52:02.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.652+0000 7f197d09a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f19781137d0 con 0x7f19781019f0 2024-05-03T20:52:02.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.652+0000 7f197659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19781023b0 0x7f197819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:52:02.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.652+0000 7f197659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19781023b0 0x7f197819fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49228/0 (socket says 172.21.15.181:49228) 2024-05-03T20:52:02.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.653+0000 7f197659c700 1 -- 172.21.15.181:0/3551870501 learned_addr learned my addr 172.21.15.181:0/3551870501 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:52:02.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.653+0000 7f1976d9d700 1 --2- 172.21.15.181:0/3551870501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19781019f0 0x7f197819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:52:02.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.653+0000 7f197759e700 1 --2- 172.21.15.181:0/3551870501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1978109570 0x7f1978199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:52:02.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.653+0000 7f197659c700 1 -- 172.21.15.181:0/3551870501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19781019f0 msgr2=0x7f197819f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:52:02.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.653+0000 7f197659c700 1 --2- 172.21.15.181:0/3551870501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19781019f0 0x7f197819f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:02.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.653+0000 7f197659c700 1 -- 172.21.15.181:0/3551870501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1978109570 msgr2=0x7f1978199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:52:02.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.653+0000 7f197659c700 1 --2- 172.21.15.181:0/3551870501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1978109570 0x7f1978199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:02.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.653+0000 7f197659c700 1 -- 172.21.15.181:0/3551870501 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1978199fd0 con 0x7f19781023b0 2024-05-03T20:52:02.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.653+0000 7f197659c700 1 --2- 172.21.15.181:0/3551870501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19781023b0 0x7f197819fbf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f196c00c9b0 tx=0x7f196c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:52:02.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.653+0000 7f195ffff700 1 -- 172.21.15.181:0/3551870501 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c007b00 con 0x7f19781023b0 2024-05-03T20:52:02.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.653+0000 7f195ffff700 1 -- 172.21.15.181:0/3551870501 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f196c007ca0 con 0x7f19781023b0 2024-05-03T20:52:02.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.653+0000 7f197d09a700 1 -- 172.21.15.181:0/3551870501 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f197819a2c0 con 0x7f19781023b0 2024-05-03T20:52:02.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.653+0000 7f195ffff700 1 -- 172.21.15.181:0/3551870501 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c018660 con 0x7f19781023b0 2024-05-03T20:52:02.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.653+0000 7f197d09a700 1 -- 172.21.15.181:0/3551870501 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19781a6580 con 0x7f19781023b0 2024-05-03T20:52:02.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.654+0000 7f197d09a700 1 -- 172.21.15.181:0/3551870501 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f197810ba00 con 0x7f19781023b0 2024-05-03T20:52:02.657 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.656+0000 7f195ffff700 1 -- 172.21.15.181:0/3551870501 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f196c00f450 con 0x7f19781023b0 2024-05-03T20:52:02.657 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.657+0000 7f195ffff700 1 --2- 172.21.15.181:0/3551870501 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f19600b2910 0x7f19600b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:52:02.657 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.657+0000 7f1976d9d700 1 --2- 172.21.15.181:0/3551870501 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f19600b2910 0x7f19600b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:52:02.658 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.657+0000 7f195ffff700 1 -- 172.21.15.181:0/3551870501 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f196c0e2640 con 0x7f19781023b0 2024-05-03T20:52:02.658 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.657+0000 7f1976d9d700 1 --2- 172.21.15.181:0/3551870501 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f19600b2910 0x7f19600b4dd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f197819a9b0 tx=0x7f1968009610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:52:02.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.660+0000 7f195ffff700 1 -- 172.21.15.181:0/3551870501 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f196c014030 con 0x7f19781023b0 2024-05-03T20:52:02.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.869+0000 7f197d09a700 1 -- 172.21.15.181:0/3551870501 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f197810b3e0 con 0x7f19781023b0 2024-05-03T20:52:02.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.871+0000 7f195ffff700 1 -- 172.21.15.181:0/3551870501 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22710 (secure 0 0 0) 0x7f196c0aa290 con 0x7f19781023b0 2024-05-03T20:52:02.870 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-03T20:52:02.870 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":83,"fsid":"1cd058ee-098b-11ef-bc97-c7b262605968","created":"2024-05-03T20:25:39.427815+0000","modified":"2024-05-03T20:34:35.581238+0000","last_up_change":"2024-05-03T20:32:38.854730+0000","last_in_change":"2024-05-03T20:32:18.787388+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T20:29:01.855187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T20:33:35.737273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T20:33:39.337556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5f594d97-10a4-4f77-aebc-c00dbe92a4af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6803","nonce":1683973278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6805","nonce":1683973278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6809","nonce":1683973278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1683973278},{"type":"v1","addr":"172.21.15.181:6807","nonce":1683973278}]},"public_addr":"172.21.15.181:6803/1683973278","cluster_addr":"172.21.15.181:6805/1683973278","heartbeat_back_addr":"172.21.15.181:6809/1683973278","heartbeat_front_addr":"172.21.15.181:6807/1683973278","state":["exists","up"]},{"osd":1,"uuid":"9726d641-8930-4b45-9192-6ee2b6bfcdcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6801","nonce":2136873375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6803","nonce":2136873375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6807","nonce":2136873375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2136873375},{"type":"v1","addr":"172.21.15.196:6805","nonce":2136873375}]},"public_addr":"172.21.15.196:6801/2136873375","cluster_addr":"172.21.15.196:6803/2136873375","heartbeat_back_addr":"172.21.15.196:6807/2136873375","heartbeat_front_addr":"172.21.15.196:6805/2136873375","state":["exists","up"]},{"osd":2,"uuid":"3d5573cc-37c1-4c3b-a61e-2363ef3ff964","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6801","nonce":3291673137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6803","nonce":3291673137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6807","nonce":3291673137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3291673137},{"type":"v1","addr":"172.21.15.204:6805","nonce":3291673137}]},"public_addr":"172.21.15.204:6801/3291673137","cluster_addr":"172.21.15.204:6803/3291673137","heartbeat_back_addr":"172.21.15.204:6807/3291673137","heartbeat_front_addr":"172.21.15.204:6805/3291673137","state":["exists","up"]},{"osd":3,"uuid":"b9b205e3-a335-4771-9a66-55cfe6a8c31a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6811","nonce":29064971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6813","nonce":29064971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6817","nonce":29064971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":29064971},{"type":"v1","addr":"172.21.15.181:6815","nonce":29064971}]},"public_addr":"172.21.15.181:6811/29064971","cluster_addr":"172.21.15.181:6813/29064971","heartbeat_back_addr":"172.21.15.181:6817/29064971","heartbeat_front_addr":"172.21.15.181:6815/29064971","state":["exists","up"]},{"osd":4,"uuid":"550f01ef-1947-4f9f-b759-34e3ea831839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6809","nonce":1145408995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6811","nonce":1145408995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6815","nonce":1145408995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1145408995},{"type":"v1","addr":"172.21.15.196:6813","nonce":1145408995}]},"public_addr":"172.21.15.196:6809/1145408995","cluster_addr":"172.21.15.196:6811/1145408995","heartbeat_back_addr":"172.21.15.196:6815/1145408995","heartbeat_front_addr":"172.21.15.196:6813/1145408995","state":["exists","up"]},{"osd":5,"uuid":"6b74cb3a-f39a-4131-8793-1d5286856aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6809","nonce":855680559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6811","nonce":855680559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6815","nonce":855680559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":855680559},{"type":"v1","addr":"172.21.15.204:6813","nonce":855680559}]},"public_addr":"172.21.15.204:6809/855680559","cluster_addr":"172.21.15.204:6811/855680559","heartbeat_back_addr":"172.21.15.204:6815/855680559","heartbeat_front_addr":"172.21.15.204:6813/855680559","state":["exists","up"]},{"osd":6,"uuid":"dbe19bdb-08eb-40aa-842b-0710963d4497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6819","nonce":1561969338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6821","nonce":1561969338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6825","nonce":1561969338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1561969338},{"type":"v1","addr":"172.21.15.181:6823","nonce":1561969338}]},"public_addr":"172.21.15.181:6819/1561969338","cluster_addr":"172.21.15.181:6821/1561969338","heartbeat_back_addr":"172.21.15.181:6825/1561969338","heartbeat_front_addr":"172.21.15.181:6823/1561969338","state":["exists","up"]},{"osd":7,"uuid":"016c145c-ad67-42ac-9775-ca2840790abc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6817","nonce":824890163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6819","nonce":824890163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6823","nonce":824890163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":824890163},{"type":"v1","addr":"172.21.15.196:6821","nonce":824890163}]},"public_addr":"172.21.15.196:6817/824890163","cluster_addr":"172.21.15.196:6819/824890163","heartbeat_back_addr":"172.21.15.196:6823/824890163","heartbeat_front_addr":"172.21.15.196:6821/824890163","state":["exists","up"]},{"osd":8,"uuid":"10235638-c1f1-41d3-8e24-f8b2344bfeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6817","nonce":3566909552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6819","nonce":3566909552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6823","nonce":3566909552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3566909552},{"type":"v1","addr":"172.21.15.204:6821","nonce":3566909552}]},"public_addr":"172.21.15.204:6817/3566909552","cluster_addr":"172.21.15.204:6819/3566909552","heartbeat_back_addr":"172.21.15.204:6823/3566909552","heartbeat_front_addr":"172.21.15.204:6821/3566909552","state":["exists","up"]},{"osd":9,"uuid":"96ce5aa6-e6a9-4d83-8562-589ac94c8cd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6827","nonce":241693472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6829","nonce":241693472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6833","nonce":241693472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":241693472},{"type":"v1","addr":"172.21.15.181:6831","nonce":241693472}]},"public_addr":"172.21.15.181:6827/241693472","cluster_addr":"172.21.15.181:6829/241693472","heartbeat_back_addr":"172.21.15.181:6833/241693472","heartbeat_front_addr":"172.21.15.181:6831/241693472","state":["exists","up"]},{"osd":10,"uuid":"d6c9ffab-731c-46d5-88cd-5f21ba5acfa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6825","nonce":3980645995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6827","nonce":3980645995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6831","nonce":3980645995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3980645995},{"type":"v1","addr":"172.21.15.196:6829","nonce":3980645995}]},"public_addr":"172.21.15.196:6825/3980645995","cluster_addr":"172.21.15.196:6827/3980645995","heartbeat_back_addr":"172.21.15.196:6831/3980645995","heartbeat_front_addr":"172.21.15.196:6829/3980645995","state":["exists","up"]},{"osd":11,"uuid":"3f58a2d6-bbbb-4e4d-bf83-2c886dd9484b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6825","nonce":488710574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6827","nonce":488710574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6831","nonce":488710574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":488710574},{"type":"v1","addr":"172.21.15.204:6829","nonce":488710574}]},"public_addr":"172.21.15.204:6825/488710574","cluster_addr":"172.21.15.204:6827/488710574","heartbeat_back_addr":"172.21.15.204:6831/488710574","heartbeat_front_addr":"172.21.15.204:6829/488710574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:28:09.700091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:28:33.838568+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:28:57.265536+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:29:21.720066+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:29:45.480447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:30:09.558384+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:30:34.253331+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:30:57.986111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:31:22.470746+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:31:47.548762+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:32:11.285657+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T20:32:36.888004+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":6},{"from":7,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6800/2389231557":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/3398276284":"2024-05-04T20:26:06.561713+0000","172.21.15.181:6801/2389231557":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/2327733609":"2024-05-04T20:26:06.561713+0000","172.21.15.181:0/628540759":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/3387163329":"2024-05-04T20:26:06.561713+0000","172.21.15.181:6800/1802984663":"2024-05-04T20:26:06.561713+0000","172.21.15.181:0/67924109":"2024-05-04T20:26:33.766772+0000","172.21.15.181:0/3693389657":"2024-05-04T20:26:33.766772+0000","172.21.15.181:6801/1802984663":"2024-05-04T20:26:06.561713+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T20:52:02.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.873+0000 7f197d09a700 1 -- 172.21.15.181:0/3551870501 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f19600b2910 msgr2=0x7f19600b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:52:02.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.873+0000 7f197d09a700 1 --2- 172.21.15.181:0/3551870501 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f19600b2910 0x7f19600b4dd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f197819a9b0 tx=0x7f1968009610 comp rx=0 tx=0).stop 2024-05-03T20:52:02.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.873+0000 7f197d09a700 1 -- 172.21.15.181:0/3551870501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19781023b0 msgr2=0x7f197819fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:52:02.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.873+0000 7f197d09a700 1 --2- 172.21.15.181:0/3551870501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19781023b0 0x7f197819fbf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f196c00c9b0 tx=0x7f196c00ccc0 comp rx=0 tx=0).stop 2024-05-03T20:52:02.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.873+0000 7f197d09a700 1 -- 172.21.15.181:0/3551870501 shutdown_connections 2024-05-03T20:52:02.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.873+0000 7f197d09a700 1 --2- 172.21.15.181:0/3551870501 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f19600b2910 0x7f19600b4dd0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:02.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.873+0000 7f197d09a700 1 --2- 172.21.15.181:0/3551870501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1978109570 0x7f1978199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:02.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.873+0000 7f197d09a700 1 --2- 172.21.15.181:0/3551870501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19781019f0 0x7f197819f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:02.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.873+0000 7f197d09a700 1 --2- 172.21.15.181:0/3551870501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19781023b0 0x7f197819fbf0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:02.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.873+0000 7f197d09a700 1 -- 172.21.15.181:0/3551870501 >> 172.21.15.181:0/3551870501 conn(0x7f19780789a0 msgr2=0x7f1978100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:52:02.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.873+0000 7f197d09a700 1 -- 172.21.15.181:0/3551870501 shutdown_connections 2024-05-03T20:52:02.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:02.874+0000 7f197d09a700 1 -- 172.21.15.181:0/3551870501 wait complete. 2024-05-03T20:52:03.204 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:02 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3551870501' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T20:52:03.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:02 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3551870501' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T20:52:03.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:02 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3551870501' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T20:52:03.499 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs fail cephfs 2024-05-03T20:52:03.740 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:52:04.167 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:03 smithi181 ceph-mon[29801]: pgmap v837: 97 pgs: 97 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-03T20:52:04.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:03 smithi196 ceph-mon[32858]: pgmap v837: 97 pgs: 97 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-03T20:52:04.362 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:03 smithi204 ceph-mon[33595]: pgmap v837: 97 pgs: 97 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-03T20:52:05.319 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.317+0000 7f839636d700 1 -- 172.21.15.181:0/2772997666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8390101c50 msgr2=0x7f8390102030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:52:05.319 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.317+0000 7f839636d700 1 --2- 172.21.15.181:0/2772997666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8390101c50 0x7f8390102030 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f83800099c0 tx=0x7f8380009cd0 comp rx=0 tx=0).stop 2024-05-03T20:52:05.319 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.317+0000 7f839636d700 1 -- 172.21.15.181:0/2772997666 shutdown_connections 2024-05-03T20:52:05.319 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.317+0000 7f839636d700 1 --2- 172.21.15.181:0/2772997666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8390108890 0x7f8390110410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:05.319 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.317+0000 7f839636d700 1 --2- 172.21.15.181:0/2772997666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8390101c50 0x7f8390102030 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:05.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.317+0000 7f839636d700 1 --2- 172.21.15.181:0/2772997666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8390102600 0x7f8390108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:05.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.317+0000 7f839636d700 1 -- 172.21.15.181:0/2772997666 >> 172.21.15.181:0/2772997666 conn(0x7f83900789a0 msgr2=0x7f83900ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:52:05.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.318+0000 7f839636d700 1 -- 172.21.15.181:0/2772997666 shutdown_connections 2024-05-03T20:52:05.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.318+0000 7f839636d700 1 -- 172.21.15.181:0/2772997666 wait complete. 2024-05-03T20:52:05.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.318+0000 7f839636d700 1 Processor -- start 2024-05-03T20:52:05.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.318+0000 7f839636d700 1 -- start start 2024-05-03T20:52:05.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.319+0000 7f839636d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8390101c50 0x7f839019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:52:05.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.319+0000 7f839636d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8390102600 0x7f839019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:52:05.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.319+0000 7f839636d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8390108890 0x7f8390199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:52:05.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.319+0000 7f839636d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8390113660 con 0x7f8390101c50 2024-05-03T20:52:05.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.319+0000 7f839636d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f83901134e0 con 0x7f8390108890 2024-05-03T20:52:05.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.319+0000 7f839636d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f83901137e0 con 0x7f8390102600 2024-05-03T20:52:05.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.319+0000 7f838ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8390101c50 0x7f839019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:52:05.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.319+0000 7f838ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8390101c50 0x7f839019f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49252/0 (socket says 172.21.15.181:49252) 2024-05-03T20:52:05.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.319+0000 7f838ffff700 1 -- 172.21.15.181:0/3381788274 learned_addr learned my addr 172.21.15.181:0/3381788274 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:52:05.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.319+0000 7f838f7fe700 1 --2- 172.21.15.181:0/3381788274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8390102600 0x7f839019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:52:05.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.319+0000 7f839490a700 1 --2- 172.21.15.181:0/3381788274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8390108890 0x7f8390199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:52:05.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.319+0000 7f838ffff700 1 -- 172.21.15.181:0/3381788274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8390102600 msgr2=0x7f839019fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:52:05.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.319+0000 7f838ffff700 1 --2- 172.21.15.181:0/3381788274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8390102600 0x7f839019fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:05.324 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.319+0000 7f838ffff700 1 -- 172.21.15.181:0/3381788274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8390108890 msgr2=0x7f8390199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:52:05.324 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.319+0000 7f838ffff700 1 --2- 172.21.15.181:0/3381788274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8390108890 0x7f8390199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:05.324 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.319+0000 7f838ffff700 1 -- 172.21.15.181:0/3381788274 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839019a080 con 0x7f8390101c50 2024-05-03T20:52:05.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.319+0000 7f838ffff700 1 --2- 172.21.15.181:0/3381788274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8390101c50 0x7f839019f710 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8380006010 tx=0x7f8380004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:52:05.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.319+0000 7f838d7fa700 1 -- 172.21.15.181:0/3381788274 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8380026070 con 0x7f8390101c50 2024-05-03T20:52:05.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.319+0000 7f839636d700 1 -- 172.21.15.181:0/3381788274 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f839019a310 con 0x7f8390101c50 2024-05-03T20:52:05.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.319+0000 7f838d7fa700 1 -- 172.21.15.181:0/3381788274 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f838000f460 con 0x7f8390101c50 2024-05-03T20:52:05.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.319+0000 7f838d7fa700 1 -- 172.21.15.181:0/3381788274 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8380009e60 con 0x7f8390101c50 2024-05-03T20:52:05.326 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.319+0000 7f839636d700 1 -- 172.21.15.181:0/3381788274 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83901a65e0 con 0x7f8390101c50 2024-05-03T20:52:05.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.320+0000 7f839636d700 1 -- 172.21.15.181:0/3381788274 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8390104d10 con 0x7f8390101c50 2024-05-03T20:52:05.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.322+0000 7f838d7fa700 1 -- 172.21.15.181:0/3381788274 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f8380029070 con 0x7f8390101c50 2024-05-03T20:52:05.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.323+0000 7f838d7fa700 1 --2- 172.21.15.181:0/3381788274 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f83780b2910 0x7f83780b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:52:05.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.323+0000 7f838f7fe700 1 --2- 172.21.15.181:0/3381788274 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f83780b2910 0x7f83780b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:52:05.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.323+0000 7f838d7fa700 1 -- 172.21.15.181:0/3381788274 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8227+0+0 (secure 0 0 0) 0x7f83800e2e50 con 0x7f8390101c50 2024-05-03T20:52:05.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.323+0000 7f838f7fe700 1 --2- 172.21.15.181:0/3381788274 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f83780b2910 0x7f83780b4dd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8390106000 tx=0x7f838400b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:52:05.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.326+0000 7f838d7fa700 1 -- 172.21.15.181:0/3381788274 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8380030080 con 0x7f8390101c50 2024-05-03T20:52:05.576 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.577+0000 7f839636d700 1 -- 172.21.15.181:0/3381788274 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f839002e510 con 0x7f8390101c50 2024-05-03T20:52:05.578 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.579+0000 7f838d7fa700 1 -- 172.21.15.181:0/3381788274 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) ==== 161+0+0 (secure 0 0 0) 0x7f8380013070 con 0x7f8390101c50 2024-05-03T20:52:05.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.583+0000 7f839636d700 1 -- 172.21.15.181:0/3381788274 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f83780b2910 msgr2=0x7f83780b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:52:05.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.583+0000 7f839636d700 1 --2- 172.21.15.181:0/3381788274 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f83780b2910 0x7f83780b4dd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8390106000 tx=0x7f838400b5b0 comp rx=0 tx=0).stop 2024-05-03T20:52:05.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.583+0000 7f839636d700 1 -- 172.21.15.181:0/3381788274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8390101c50 msgr2=0x7f839019f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:52:05.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.583+0000 7f839636d700 1 --2- 172.21.15.181:0/3381788274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8390101c50 0x7f839019f710 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8380006010 tx=0x7f8380004690 comp rx=0 tx=0).stop 2024-05-03T20:52:05.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.583+0000 7f839636d700 1 -- 172.21.15.181:0/3381788274 shutdown_connections 2024-05-03T20:52:05.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.583+0000 7f839636d700 1 --2- 172.21.15.181:0/3381788274 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f83780b2910 0x7f83780b4dd0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:05.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.583+0000 7f839636d700 1 --2- 172.21.15.181:0/3381788274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8390108890 0x7f8390199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:05.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.583+0000 7f839636d700 1 --2- 172.21.15.181:0/3381788274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8390101c50 0x7f839019f710 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:05.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.583+0000 7f839636d700 1 --2- 172.21.15.181:0/3381788274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8390102600 0x7f839019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:05.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.583+0000 7f839636d700 1 -- 172.21.15.181:0/3381788274 >> 172.21.15.181:0/3381788274 conn(0x7f83900789a0 msgr2=0x7f83900ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:52:05.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.583+0000 7f839636d700 1 -- 172.21.15.181:0/3381788274 shutdown_connections 2024-05-03T20:52:05.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:05.583+0000 7f839636d700 1 -- 172.21.15.181:0/3381788274 wait complete. 2024-05-03T20:52:05.584 INFO:teuthology.orchestra.run.smithi181.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-03T20:52:05.619 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:05 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-a[29797]: 2024-05-03T20:52:05.578+0000 7f280b45f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-03T20:52:05.903 INFO:journalctl@ceph.mds.g.smithi181.stdout:May 03 20:52:05 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-g[71911]: ignoring --setuser ceph since I am not root 2024-05-03T20:52:05.904 INFO:journalctl@ceph.mds.g.smithi181.stdout:May 03 20:52:05 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-g[71911]: ignoring --setgroup ceph since I am not root 2024-05-03T20:52:05.904 INFO:journalctl@ceph.mds.g.smithi181.stdout:May 03 20:52:05 smithi181 ceph-mds[71915]: -- 172.21.15.181:0/2936554682 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c70282da40 con 0x55c702800800 2024-05-03T20:52:05.904 INFO:journalctl@ceph.mds.g.smithi181.stdout:May 03 20:52:05 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-g[71911]: starting mds.g at 2024-05-03T20:52:05.918 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 03 20:52:05 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-i[64570]: ignoring --setuser ceph since I am not root 2024-05-03T20:52:05.919 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 03 20:52:05 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-i[64570]: ignoring --setgroup ceph since I am not root 2024-05-03T20:52:05.919 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 03 20:52:05 smithi204 ceph-mds[64574]: -- 172.21.15.204:0/3898090210 <== mon.0 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562a7732da40 con 0x562a77300800 2024-05-03T20:52:05.919 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 03 20:52:05 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-i[64570]: starting mds.i at 2024-05-03T20:52:05.920 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 03 20:52:05 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-c[64248]: ignoring --setuser ceph since I am not root 2024-05-03T20:52:05.920 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 03 20:52:05 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-c[64248]: ignoring --setgroup ceph since I am not root 2024-05-03T20:52:05.920 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 03 20:52:05 smithi204 ceph-mds[64252]: -- 172.21.15.204:0/1795405466 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aeb369da40 con 0x55aeb3671000 2024-05-03T20:52:05.920 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 03 20:52:05 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-c[64248]: starting mds.c at 2024-05-03T20:52:05.920 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 03 20:52:05 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-f[63943]: ignoring --setuser ceph since I am not root 2024-05-03T20:52:05.921 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 03 20:52:05 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-f[63943]: ignoring --setgroup ceph since I am not root 2024-05-03T20:52:05.921 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 03 20:52:05 smithi204 ceph-mds[63947]: -- 172.21.15.204:0/1724657272 <== mon.0 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564385beda40 con 0x564385bc0c00 2024-05-03T20:52:05.921 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 03 20:52:05 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-f[63943]: starting mds.f at 2024-05-03T20:52:05.921 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 03 20:52:05 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-l[63631]: ignoring --setuser ceph since I am not root 2024-05-03T20:52:05.921 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 03 20:52:05 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-l[63631]: ignoring --setgroup ceph since I am not root 2024-05-03T20:52:05.921 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 03 20:52:05 smithi204 ceph-mds[63635]: -- 172.21.15.204:0/4179671840 <== mon.2 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5568feedba40 con 0x5568feeaf000 2024-05-03T20:52:05.921 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 03 20:52:05 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-l[63631]: starting mds.l at 2024-05-03T20:52:05.922 INFO:journalctl@ceph.mds.e.smithi196.stdout:May 03 20:52:05 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-e[62924]: ignoring --setuser ceph since I am not root 2024-05-03T20:52:05.922 INFO:journalctl@ceph.mds.e.smithi196.stdout:May 03 20:52:05 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-e[62924]: ignoring --setgroup ceph since I am not root 2024-05-03T20:52:05.922 INFO:journalctl@ceph.mds.e.smithi196.stdout:May 03 20:52:05 smithi196 ceph-mds[62928]: -- 172.21.15.196:0/1243386962 <== mon.2 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560abcffba40 con 0x560abcfcec00 2024-05-03T20:52:05.922 INFO:journalctl@ceph.mds.e.smithi196.stdout:May 03 20:52:05 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-e[62924]: starting mds.e at 2024-05-03T20:52:05.923 INFO:journalctl@ceph.mds.h.smithi196.stdout:May 03 20:52:05 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-h[63236]: ignoring --setuser ceph since I am not root 2024-05-03T20:52:05.923 INFO:journalctl@ceph.mds.h.smithi196.stdout:May 03 20:52:05 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-h[63236]: ignoring --setgroup ceph since I am not root 2024-05-03T20:52:05.923 INFO:journalctl@ceph.mds.h.smithi196.stdout:May 03 20:52:05 smithi196 ceph-mds[63240]: -- 172.21.15.196:0/1138074214 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55abdf589a40 con 0x55abdf55cc00 2024-05-03T20:52:05.923 INFO:journalctl@ceph.mds.h.smithi196.stdout:May 03 20:52:05 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-h[63236]: starting mds.h at 2024-05-03T20:52:05.924 INFO:journalctl@ceph.mds.k.smithi196.stdout:May 03 20:52:05 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-k[62601]: ignoring --setuser ceph since I am not root 2024-05-03T20:52:05.924 INFO:journalctl@ceph.mds.k.smithi196.stdout:May 03 20:52:05 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-k[62601]: ignoring --setgroup ceph since I am not root 2024-05-03T20:52:05.924 INFO:journalctl@ceph.mds.k.smithi196.stdout:May 03 20:52:05 smithi196 ceph-mds[62605]: -- 172.21.15.196:0/2784915327 <== mon.0 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56077231da40 con 0x5607722f0800 2024-05-03T20:52:05.924 INFO:journalctl@ceph.mds.k.smithi196.stdout:May 03 20:52:05 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-k[62601]: starting mds.k at 2024-05-03T20:52:05.924 INFO:journalctl@ceph.mds.b.smithi196.stdout:May 03 20:52:05 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-b[63557]: ignoring --setuser ceph since I am not root 2024-05-03T20:52:05.924 INFO:journalctl@ceph.mds.b.smithi196.stdout:May 03 20:52:05 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-b[63557]: ignoring --setgroup ceph since I am not root 2024-05-03T20:52:05.925 INFO:journalctl@ceph.mds.b.smithi196.stdout:May 03 20:52:05 smithi196 ceph-mds[63562]: -- 172.21.15.196:0/389441217 <== mon.0 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558f8dc03a40 con 0x558f8dbd7000 2024-05-03T20:52:05.925 INFO:journalctl@ceph.mds.b.smithi196.stdout:May 03 20:52:05 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-b[63557]: starting mds.b at 2024-05-03T20:52:06.187 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:05 smithi181 ceph-mon[29801]: pgmap v838: 97 pgs: 97 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-03T20:52:06.188 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:05 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3381788274' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-03T20:52:06.188 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:05 smithi181 ceph-mon[29801]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-03T20:52:06.188 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:05 smithi181 ceph-mon[29801]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-03T20:52:06.188 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:05 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3381788274' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-03T20:52:06.188 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:05 smithi181 ceph-mon[29801]: osdmap e84: 12 total, 12 up, 12 in 2024-05-03T20:52:06.188 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:05 smithi181 ceph-mon[29801]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-03T20:52:06.215 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-03T20:52:06.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:05 smithi196 ceph-mon[32858]: pgmap v838: 97 pgs: 97 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-03T20:52:06.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:05 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3381788274' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-03T20:52:06.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:05 smithi196 ceph-mon[32858]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-03T20:52:06.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:05 smithi196 ceph-mon[32858]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-03T20:52:06.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:05 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3381788274' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-03T20:52:06.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:05 smithi196 ceph-mon[32858]: osdmap e84: 12 total, 12 up, 12 in 2024-05-03T20:52:06.357 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:05 smithi196 ceph-mon[32858]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-03T20:52:06.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:05 smithi204 ceph-mon[33595]: pgmap v838: 97 pgs: 97 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-03T20:52:06.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:05 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3381788274' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-03T20:52:06.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:05 smithi204 ceph-mon[33595]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-03T20:52:06.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:05 smithi204 ceph-mon[33595]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-03T20:52:06.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:05 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3381788274' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-03T20:52:06.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:05 smithi204 ceph-mon[33595]: osdmap e84: 12 total, 12 up, 12 in 2024-05-03T20:52:06.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:05 smithi204 ceph-mon[33595]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-03T20:52:06.473 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:52:07.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:07 smithi181 ceph-mon[29801]: pgmap v840: 97 pgs: 97 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 11 op/s 2024-05-03T20:52:07.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:07 smithi181 ceph-mon[29801]: mds.? [v2:172.21.15.181:6838/1905293031,v1:172.21.15.181:6839/1905293031] up:boot 2024-05-03T20:52:07.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:07 smithi181 ceph-mon[29801]: mds.? [v2:172.21.15.204:6833/1585433286,v1:172.21.15.204:6835/1585433286] up:boot 2024-05-03T20:52:07.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:07 smithi181 ceph-mon[29801]: mds.? [v2:172.21.15.204:6832/2950446073,v1:172.21.15.204:6834/2950446073] up:boot 2024-05-03T20:52:07.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:07 smithi181 ceph-mon[29801]: mds.? [v2:172.21.15.181:6840/661085964,v1:172.21.15.181:6841/661085964] up:boot 2024-05-03T20:52:07.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:07 smithi181 ceph-mon[29801]: mds.? [v2:172.21.15.196:6832/3425829080,v1:172.21.15.196:6836/3425829080] up:boot 2024-05-03T20:52:07.713 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:07 smithi181 ceph-mon[29801]: mds.? [v2:172.21.15.196:6834/1309195128,v1:172.21.15.196:6838/1309195128] up:boot 2024-05-03T20:52:07.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:07 smithi181 ceph-mon[29801]: mds.? [v2:172.21.15.196:6833/959630895,v1:172.21.15.196:6837/959630895] up:boot 2024-05-03T20:52:07.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:07 smithi181 ceph-mon[29801]: mds.? [v2:172.21.15.196:6835/3652942075,v1:172.21.15.196:6839/3652942075] up:boot 2024-05-03T20:52:07.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:07 smithi181 ceph-mon[29801]: mds.? [v2:172.21.15.204:6836/2258295000,v1:172.21.15.204:6838/2258295000] up:boot 2024-05-03T20:52:07.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:07 smithi181 ceph-mon[29801]: mds.? [v2:172.21.15.204:6837/93351356,v1:172.21.15.204:6839/93351356] up:boot 2024-05-03T20:52:07.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:07 smithi181 ceph-mon[29801]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-03T20:52:07.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-03T20:52:07.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-03T20:52:07.714 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-03T20:52:07.715 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T20:52:07.715 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-03T20:52:07.715 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T20:52:07.715 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T20:52:07.715 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T20:52:07.715 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T20:52:07.715 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:07 smithi181 ceph-mon[29801]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T20:52:07.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:07 smithi196 ceph-mon[32858]: pgmap v840: 97 pgs: 97 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 11 op/s 2024-05-03T20:52:07.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:07 smithi196 ceph-mon[32858]: mds.? [v2:172.21.15.181:6838/1905293031,v1:172.21.15.181:6839/1905293031] up:boot 2024-05-03T20:52:07.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:07 smithi196 ceph-mon[32858]: mds.? [v2:172.21.15.204:6833/1585433286,v1:172.21.15.204:6835/1585433286] up:boot 2024-05-03T20:52:07.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:07 smithi196 ceph-mon[32858]: mds.? [v2:172.21.15.204:6832/2950446073,v1:172.21.15.204:6834/2950446073] up:boot 2024-05-03T20:52:07.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:07 smithi196 ceph-mon[32858]: mds.? [v2:172.21.15.181:6840/661085964,v1:172.21.15.181:6841/661085964] up:boot 2024-05-03T20:52:07.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:07 smithi196 ceph-mon[32858]: mds.? [v2:172.21.15.196:6832/3425829080,v1:172.21.15.196:6836/3425829080] up:boot 2024-05-03T20:52:07.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:07 smithi196 ceph-mon[32858]: mds.? [v2:172.21.15.196:6834/1309195128,v1:172.21.15.196:6838/1309195128] up:boot 2024-05-03T20:52:07.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:07 smithi196 ceph-mon[32858]: mds.? [v2:172.21.15.196:6833/959630895,v1:172.21.15.196:6837/959630895] up:boot 2024-05-03T20:52:07.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:07 smithi196 ceph-mon[32858]: mds.? [v2:172.21.15.196:6835/3652942075,v1:172.21.15.196:6839/3652942075] up:boot 2024-05-03T20:52:07.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:07 smithi196 ceph-mon[32858]: mds.? [v2:172.21.15.204:6836/2258295000,v1:172.21.15.204:6838/2258295000] up:boot 2024-05-03T20:52:07.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:07 smithi196 ceph-mon[32858]: mds.? [v2:172.21.15.204:6837/93351356,v1:172.21.15.204:6839/93351356] up:boot 2024-05-03T20:52:07.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:07 smithi196 ceph-mon[32858]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-03T20:52:07.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-03T20:52:07.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-03T20:52:07.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-03T20:52:07.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T20:52:07.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-03T20:52:07.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T20:52:07.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T20:52:07.859 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T20:52:07.859 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T20:52:07.859 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:07 smithi196 ceph-mon[32858]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T20:52:07.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:07 smithi204 ceph-mon[33595]: pgmap v840: 97 pgs: 97 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 11 op/s 2024-05-03T20:52:07.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:07 smithi204 ceph-mon[33595]: mds.? [v2:172.21.15.181:6838/1905293031,v1:172.21.15.181:6839/1905293031] up:boot 2024-05-03T20:52:07.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:07 smithi204 ceph-mon[33595]: mds.? [v2:172.21.15.204:6833/1585433286,v1:172.21.15.204:6835/1585433286] up:boot 2024-05-03T20:52:07.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:07 smithi204 ceph-mon[33595]: mds.? [v2:172.21.15.204:6832/2950446073,v1:172.21.15.204:6834/2950446073] up:boot 2024-05-03T20:52:07.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:07 smithi204 ceph-mon[33595]: mds.? [v2:172.21.15.181:6840/661085964,v1:172.21.15.181:6841/661085964] up:boot 2024-05-03T20:52:07.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:07 smithi204 ceph-mon[33595]: mds.? [v2:172.21.15.196:6832/3425829080,v1:172.21.15.196:6836/3425829080] up:boot 2024-05-03T20:52:07.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:07 smithi204 ceph-mon[33595]: mds.? [v2:172.21.15.196:6834/1309195128,v1:172.21.15.196:6838/1309195128] up:boot 2024-05-03T20:52:07.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:07 smithi204 ceph-mon[33595]: mds.? [v2:172.21.15.196:6833/959630895,v1:172.21.15.196:6837/959630895] up:boot 2024-05-03T20:52:07.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:07 smithi204 ceph-mon[33595]: mds.? [v2:172.21.15.196:6835/3652942075,v1:172.21.15.196:6839/3652942075] up:boot 2024-05-03T20:52:07.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:07 smithi204 ceph-mon[33595]: mds.? [v2:172.21.15.204:6836/2258295000,v1:172.21.15.204:6838/2258295000] up:boot 2024-05-03T20:52:07.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:07 smithi204 ceph-mon[33595]: mds.? [v2:172.21.15.204:6837/93351356,v1:172.21.15.204:6839/93351356] up:boot 2024-05-03T20:52:07.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:07 smithi204 ceph-mon[33595]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-03T20:52:07.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-03T20:52:07.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-03T20:52:07.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-03T20:52:07.864 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T20:52:07.865 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-03T20:52:07.865 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T20:52:07.865 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T20:52:07.865 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T20:52:07.865 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T20:52:07.865 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:07 smithi204 ceph-mon[33595]: from='mgr.14152 172.21.15.181:0/3341021726' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T20:52:08.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.085+0000 7f9dd35bb700 1 -- 172.21.15.181:0/2413765801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dcc06c5b0 msgr2=0x7f9dcc110350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:52:08.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.085+0000 7f9dd35bb700 1 --2- 172.21.15.181:0/2413765801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dcc06c5b0 0x7f9dcc110350 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9dc400b0a0 tx=0x7f9dc400b3b0 comp rx=0 tx=0).stop 2024-05-03T20:52:08.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.085+0000 7f9dd35bb700 1 -- 172.21.15.181:0/2413765801 shutdown_connections 2024-05-03T20:52:08.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.085+0000 7f9dd35bb700 1 --2- 172.21.15.181:0/2413765801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dcc06c5b0 0x7f9dcc110350 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:08.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.085+0000 7f9dd35bb700 1 --2- 172.21.15.181:0/2413765801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dcc109610 0x7f9dcc1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:08.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.085+0000 7f9dd35bb700 1 --2- 172.21.15.181:0/2413765801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9dcc06bbf0 0x7f9dcc06c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:08.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.085+0000 7f9dd35bb700 1 -- 172.21.15.181:0/2413765801 >> 172.21.15.181:0/2413765801 conn(0x7f9dcc079290 msgr2=0x7f9dcc0796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:52:08.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.085+0000 7f9dd35bb700 1 -- 172.21.15.181:0/2413765801 shutdown_connections 2024-05-03T20:52:08.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.085+0000 7f9dd35bb700 1 -- 172.21.15.181:0/2413765801 wait complete. 2024-05-03T20:52:08.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.085+0000 7f9dd35bb700 1 Processor -- start 2024-05-03T20:52:08.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.085+0000 7f9dd35bb700 1 -- start start 2024-05-03T20:52:08.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.086+0000 7f9dd35bb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dcc06bbf0 0x7f9dcc102750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:52:08.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.086+0000 7f9dd35bb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9dcc06c5b0 0x7f9dcc102c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:52:08.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.086+0000 7f9dd35bb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dcc109610 0x7f9dcc1031d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:52:08.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.086+0000 7f9dd35bb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc113660 con 0x7f9dcc06bbf0 2024-05-03T20:52:08.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.086+0000 7f9dd35bb700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc1134e0 con 0x7f9dcc06c5b0 2024-05-03T20:52:08.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.086+0000 7f9dd35bb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc1137e0 con 0x7f9dcc109610 2024-05-03T20:52:08.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.086+0000 7f9dd1357700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dcc06bbf0 0x7f9dcc102750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:52:08.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.086+0000 7f9dd1357700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dcc06bbf0 0x7f9dcc102750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52018/0 (socket says 172.21.15.181:52018) 2024-05-03T20:52:08.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.086+0000 7f9dd1357700 1 -- 172.21.15.181:0/3604062324 learned_addr learned my addr 172.21.15.181:0/3604062324 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:52:08.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.086+0000 7f9dd1b58700 1 --2- 172.21.15.181:0/3604062324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dcc109610 0x7f9dcc1031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:52:08.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.086+0000 7f9dd0b56700 1 --2- 172.21.15.181:0/3604062324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9dcc06c5b0 0x7f9dcc102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:52:08.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.086+0000 7f9dd1357700 1 -- 172.21.15.181:0/3604062324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dcc109610 msgr2=0x7f9dcc1031d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:52:08.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.086+0000 7f9dd1357700 1 --2- 172.21.15.181:0/3604062324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dcc109610 0x7f9dcc1031d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:08.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.086+0000 7f9dd1357700 1 -- 172.21.15.181:0/3604062324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9dcc06c5b0 msgr2=0x7f9dcc102c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:52:08.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.086+0000 7f9dd1357700 1 --2- 172.21.15.181:0/3604062324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9dcc06c5b0 0x7f9dcc102c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:08.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.086+0000 7f9dd1357700 1 -- 172.21.15.181:0/3604062324 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dcc1a8020 con 0x7f9dcc06bbf0 2024-05-03T20:52:08.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.086+0000 7f9dd1357700 1 --2- 172.21.15.181:0/3604062324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dcc06bbf0 0x7f9dcc102750 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9dc800d950 tx=0x7f9dc800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:52:08.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.086+0000 7f9dc27fc700 1 -- 172.21.15.181:0/3604062324 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8016070 con 0x7f9dcc06bbf0 2024-05-03T20:52:08.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.087+0000 7f9dc27fc700 1 -- 172.21.15.181:0/3604062324 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dc8009df0 con 0x7f9dcc06bbf0 2024-05-03T20:52:08.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.087+0000 7f9dd35bb700 1 -- 172.21.15.181:0/3604062324 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dcc1a8310 con 0x7f9dcc06bbf0 2024-05-03T20:52:08.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.087+0000 7f9dc27fc700 1 -- 172.21.15.181:0/3604062324 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc800f3c0 con 0x7f9dcc06bbf0 2024-05-03T20:52:08.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.087+0000 7f9dd35bb700 1 -- 172.21.15.181:0/3604062324 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dcc1a8850 con 0x7f9dcc06bbf0 2024-05-03T20:52:08.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.087+0000 7f9dd35bb700 1 -- 172.21.15.181:0/3604062324 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dcc104c30 con 0x7f9dcc06bbf0 2024-05-03T20:52:08.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.089+0000 7f9dc27fc700 1 -- 172.21.15.181:0/3604062324 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f9dc8019070 con 0x7f9dcc06bbf0 2024-05-03T20:52:08.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.089+0000 7f9dc27fc700 1 --2- 172.21.15.181:0/3604062324 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f9db80b2910 0x7f9db80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:52:08.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.089+0000 7f9dd0b56700 1 --2- 172.21.15.181:0/3604062324 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f9db80b2910 0x7f9db80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:52:08.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.089+0000 7f9dc27fc700 1 -- 172.21.15.181:0/3604062324 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8657+0+0 (secure 0 0 0) 0x7f9dc80e20e0 con 0x7f9dcc06bbf0 2024-05-03T20:52:08.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.090+0000 7f9dd0b56700 1 --2- 172.21.15.181:0/3604062324 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f9db80b2910 0x7f9db80b4dd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc005950 tx=0x7f9dbc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:52:08.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.094+0000 7f9dc27fc700 1 -- 172.21.15.181:0/3604062324 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9dc80a9af0 con 0x7f9dcc06bbf0 2024-05-03T20:52:08.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.343+0000 7f9dd35bb700 1 -- 172.21.15.181:0/3604062324 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f9dcc02e510 con 0x7f9dcc06bbf0 2024-05-03T20:52:08.587 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.588+0000 7f9dc27fc700 1 -- 172.21.15.181:0/3604062324 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) ==== 104+0+0 (secure 0 0 0) 0x7f9dc80a93f0 con 0x7f9dcc06bbf0 2024-05-03T20:52:08.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.590+0000 7f9dd35bb700 1 -- 172.21.15.181:0/3604062324 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f9db80b2910 msgr2=0x7f9db80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:52:08.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.590+0000 7f9dd35bb700 1 --2- 172.21.15.181:0/3604062324 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f9db80b2910 0x7f9db80b4dd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc005950 tx=0x7f9dbc0058e0 comp rx=0 tx=0).stop 2024-05-03T20:52:08.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.590+0000 7f9dd35bb700 1 -- 172.21.15.181:0/3604062324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dcc06bbf0 msgr2=0x7f9dcc102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:52:08.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.590+0000 7f9dd35bb700 1 --2- 172.21.15.181:0/3604062324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dcc06bbf0 0x7f9dcc102750 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9dc800d950 tx=0x7f9dc800dc60 comp rx=0 tx=0).stop 2024-05-03T20:52:08.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.590+0000 7f9dd35bb700 1 -- 172.21.15.181:0/3604062324 shutdown_connections 2024-05-03T20:52:08.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.590+0000 7f9dd35bb700 1 --2- 172.21.15.181:0/3604062324 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f9db80b2910 0x7f9db80b4dd0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:08.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.590+0000 7f9dd35bb700 1 --2- 172.21.15.181:0/3604062324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dcc109610 0x7f9dcc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:08.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.590+0000 7f9dd35bb700 1 --2- 172.21.15.181:0/3604062324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dcc06bbf0 0x7f9dcc102750 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:08.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.590+0000 7f9dd35bb700 1 --2- 172.21.15.181:0/3604062324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9dcc06c5b0 0x7f9dcc102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:08.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.590+0000 7f9dd35bb700 1 -- 172.21.15.181:0/3604062324 >> 172.21.15.181:0/3604062324 conn(0x7f9dcc079290 msgr2=0x7f9dcc101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:52:08.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.591+0000 7f9dd35bb700 1 -- 172.21.15.181:0/3604062324 shutdown_connections 2024-05-03T20:52:08.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:08.591+0000 7f9dd35bb700 1 -- 172.21.15.181:0/3604062324 wait complete. 2024-05-03T20:52:08.601 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:08 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3604062324' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T20:52:08.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:08 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3604062324' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T20:52:08.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:08 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3604062324' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T20:52:09.222 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-03T20:52:09.464 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:52:09.856 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:09 smithi181 ceph-mon[29801]: pgmap v841: 97 pgs: 97 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-03T20:52:09.856 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:09 smithi181 ceph-mon[29801]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-03T20:52:09.856 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:09 smithi181 ceph-mon[29801]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-03T20:52:09.857 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:09 smithi181 ceph-mon[29801]: Cluster is now healthy 2024-05-03T20:52:09.857 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:09 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3604062324' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-03T20:52:09.857 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:09 smithi181 ceph-mon[29801]: fsmap 12 up:standby 2024-05-03T20:52:09.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:09 smithi196 ceph-mon[32858]: pgmap v841: 97 pgs: 97 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-03T20:52:09.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:09 smithi196 ceph-mon[32858]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-03T20:52:09.857 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:09 smithi196 ceph-mon[32858]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-03T20:52:09.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:09 smithi196 ceph-mon[32858]: Cluster is now healthy 2024-05-03T20:52:09.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:09 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3604062324' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-03T20:52:09.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:09 smithi196 ceph-mon[32858]: fsmap 12 up:standby 2024-05-03T20:52:09.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:09 smithi204 ceph-mon[33595]: pgmap v841: 97 pgs: 97 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-03T20:52:09.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:09 smithi204 ceph-mon[33595]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-03T20:52:09.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:09 smithi204 ceph-mon[33595]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-03T20:52:09.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:09 smithi204 ceph-mon[33595]: Cluster is now healthy 2024-05-03T20:52:09.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:09 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3604062324' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-03T20:52:09.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:09 smithi204 ceph-mon[33595]: fsmap 12 up:standby 2024-05-03T20:52:11.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.093+0000 7f2100867700 1 -- 172.21.15.181:0/4177884367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20fc103560 msgr2=0x7f20fc1039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:52:11.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.093+0000 7f2100867700 1 --2- 172.21.15.181:0/4177884367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20fc103560 0x7f20fc1039e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f20f00099c0 tx=0x7f20f0009cd0 comp rx=0 tx=0).stop 2024-05-03T20:52:11.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.093+0000 7f2100867700 1 -- 172.21.15.181:0/4177884367 shutdown_connections 2024-05-03T20:52:11.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.093+0000 7f2100867700 1 --2- 172.21.15.181:0/4177884367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20fc103f20 0x7f20fc110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:11.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.093+0000 7f2100867700 1 --2- 172.21.15.181:0/4177884367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20fc109540 0x7f20fc109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:11.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.093+0000 7f2100867700 1 --2- 172.21.15.181:0/4177884367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20fc103560 0x7f20fc1039e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:11.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.093+0000 7f2100867700 1 -- 172.21.15.181:0/4177884367 >> 172.21.15.181:0/4177884367 conn(0x7f20fc07a660 msgr2=0x7f20fc07aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:52:11.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.093+0000 7f2100867700 1 -- 172.21.15.181:0/4177884367 shutdown_connections 2024-05-03T20:52:11.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.093+0000 7f2100867700 1 -- 172.21.15.181:0/4177884367 wait complete. 2024-05-03T20:52:11.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.094+0000 7f2100867700 1 Processor -- start 2024-05-03T20:52:11.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.094+0000 7f2100867700 1 -- start start 2024-05-03T20:52:11.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.094+0000 7f2100867700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20fc103560 0x7f20fc0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:52:11.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.094+0000 7f2100867700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20fc103f20 0x7f20fc075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:52:11.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.094+0000 7f2100867700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20fc109540 0x7f20fc076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:52:11.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.094+0000 7f2100867700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f20fc113640 con 0x7f20fc109540 2024-05-03T20:52:11.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.094+0000 7f2100867700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f20fc1134c0 con 0x7f20fc103560 2024-05-03T20:52:11.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.094+0000 7f2100867700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f20fc1137c0 con 0x7f20fc103f20 2024-05-03T20:52:11.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.094+0000 7f20fad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20fc109540 0x7f20fc076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:52:11.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.094+0000 7f20fad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20fc109540 0x7f20fc076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52034/0 (socket says 172.21.15.181:52034) 2024-05-03T20:52:11.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.094+0000 7f20fad9d700 1 -- 172.21.15.181:0/369241400 learned_addr learned my addr 172.21.15.181:0/369241400 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:52:11.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.094+0000 7f20fa59c700 1 --2- 172.21.15.181:0/369241400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20fc103560 0x7f20fc0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:52:11.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.094+0000 7f20f9d9b700 1 --2- 172.21.15.181:0/369241400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20fc103f20 0x7f20fc075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:52:11.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.094+0000 7f20fad9d700 1 -- 172.21.15.181:0/369241400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20fc103f20 msgr2=0x7f20fc075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:52:11.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.094+0000 7f20fad9d700 1 --2- 172.21.15.181:0/369241400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20fc103f20 0x7f20fc075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:11.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.094+0000 7f20fad9d700 1 -- 172.21.15.181:0/369241400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20fc103560 msgr2=0x7f20fc0757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:52:11.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.094+0000 7f20fad9d700 1 --2- 172.21.15.181:0/369241400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20fc103560 0x7f20fc0757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:11.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.094+0000 7f20fad9d700 1 -- 172.21.15.181:0/369241400 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20fc1a7fe0 con 0x7f20fc109540 2024-05-03T20:52:11.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.095+0000 7f20fad9d700 1 --2- 172.21.15.181:0/369241400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20fc109540 0x7f20fc076220 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f20f4007d70 tx=0x7f20f400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:52:11.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.095+0000 7f20eb7fe700 1 -- 172.21.15.181:0/369241400 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f4016020 con 0x7f20fc109540 2024-05-03T20:52:11.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.095+0000 7f2100867700 1 -- 172.21.15.181:0/369241400 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20fc1a82d0 con 0x7f20fc109540 2024-05-03T20:52:11.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.095+0000 7f20eb7fe700 1 -- 172.21.15.181:0/369241400 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20f400ad40 con 0x7f20fc109540 2024-05-03T20:52:11.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.095+0000 7f20eb7fe700 1 -- 172.21.15.181:0/369241400 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f40115e0 con 0x7f20fc109540 2024-05-03T20:52:11.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.095+0000 7f2100867700 1 -- 172.21.15.181:0/369241400 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20fc1a8790 con 0x7f20fc109540 2024-05-03T20:52:11.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.096+0000 7f2100867700 1 -- 172.21.15.181:0/369241400 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20fc10b9d0 con 0x7f20fc109540 2024-05-03T20:52:11.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.097+0000 7f20eb7fe700 1 -- 172.21.15.181:0/369241400 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f20f4019020 con 0x7f20fc109540 2024-05-03T20:52:11.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.098+0000 7f20eb7fe700 1 --2- 172.21.15.181:0/369241400 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f20e40b2910 0x7f20e40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:52:11.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.098+0000 7f20fa59c700 1 --2- 172.21.15.181:0/369241400 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f20e40b2910 0x7f20e40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:52:11.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.098+0000 7f20eb7fe700 1 -- 172.21.15.181:0/369241400 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8657+0+0 (secure 0 0 0) 0x7f20f40e28a0 con 0x7f20fc109540 2024-05-03T20:52:11.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.098+0000 7f20fa59c700 1 --2- 172.21.15.181:0/369241400 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f20e40b2910 0x7f20e40b4dd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00bbb0 tx=0x7f20ec00b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:52:11.100 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.101+0000 7f20eb7fe700 1 -- 172.21.15.181:0/369241400 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f20f4020020 con 0x7f20fc109540 2024-05-03T20:52:11.302 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.303+0000 7f2100867700 1 -- 172.21.15.181:0/369241400 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f20fc104bb0 con 0x7f20fc109540 2024-05-03T20:52:11.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.694+0000 7f20eb7fe700 1 -- 172.21.15.181:0/369241400 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v85) ==== 181+0+0 (secure 0 0 0) 0x7f20fc104bb0 con 0x7f20fc109540 2024-05-03T20:52:11.696 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.696+0000 7f2100867700 1 -- 172.21.15.181:0/369241400 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f20e40b2910 msgr2=0x7f20e40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:52:11.696 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.696+0000 7f2100867700 1 --2- 172.21.15.181:0/369241400 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f20e40b2910 0x7f20e40b4dd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00bbb0 tx=0x7f20ec00b590 comp rx=0 tx=0).stop 2024-05-03T20:52:11.696 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.696+0000 7f2100867700 1 -- 172.21.15.181:0/369241400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20fc109540 msgr2=0x7f20fc076220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:52:11.696 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.696+0000 7f2100867700 1 --2- 172.21.15.181:0/369241400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20fc109540 0x7f20fc076220 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f20f4007d70 tx=0x7f20f400a340 comp rx=0 tx=0).stop 2024-05-03T20:52:11.696 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.696+0000 7f2100867700 1 -- 172.21.15.181:0/369241400 shutdown_connections 2024-05-03T20:52:11.696 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.696+0000 7f2100867700 1 --2- 172.21.15.181:0/369241400 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7f20e40b2910 0x7f20e40b4dd0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:11.696 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.696+0000 7f2100867700 1 --2- 172.21.15.181:0/369241400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20fc109540 0x7f20fc076220 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:11.697 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.696+0000 7f2100867700 1 --2- 172.21.15.181:0/369241400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20fc103560 0x7f20fc0757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:11.697 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.696+0000 7f2100867700 1 --2- 172.21.15.181:0/369241400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20fc103f20 0x7f20fc075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:11.697 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.696+0000 7f2100867700 1 -- 172.21.15.181:0/369241400 >> 172.21.15.181:0/369241400 conn(0x7f20fc07a660 msgr2=0x7f20fc1020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:52:11.697 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.696+0000 7f2100867700 1 -- 172.21.15.181:0/369241400 shutdown_connections 2024-05-03T20:52:11.697 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:11.696+0000 7f2100867700 1 -- 172.21.15.181:0/369241400 wait complete. 2024-05-03T20:52:11.697 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' removed 2024-05-03T20:52:11.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:11 smithi181 ceph-mon[29801]: pgmap v842: 97 pgs: 97 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-03T20:52:11.963 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:11 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/369241400' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T20:52:12.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:11 smithi196 ceph-mon[32858]: pgmap v842: 97 pgs: 97 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-03T20:52:12.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:11 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/369241400' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T20:52:12.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:11 smithi204 ceph-mon[33595]: pgmap v842: 97 pgs: 97 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-03T20:52:12.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:11 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/369241400' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T20:52:12.286 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-03T20:52:12.525 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T20:52:12.929 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:12 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/369241400' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T20:52:12.930 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:12 smithi181 ceph-mon[29801]: osdmap e85: 12 total, 12 up, 12 in 2024-05-03T20:52:13.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:12 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/369241400' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T20:52:13.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:12 smithi196 ceph-mon[32858]: osdmap e85: 12 total, 12 up, 12 in 2024-05-03T20:52:13.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:12 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/369241400' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T20:52:13.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:12 smithi204 ceph-mon[33595]: osdmap e85: 12 total, 12 up, 12 in 2024-05-03T20:52:13.878 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:13 smithi181 ceph-mon[29801]: pgmap v844: 65 pgs: 65 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:52:14.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:13 smithi196 ceph-mon[32858]: pgmap v844: 65 pgs: 65 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:52:14.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:13 smithi204 ceph-mon[33595]: pgmap v844: 65 pgs: 65 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:52:14.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.125+0000 7fb42a4cd700 1 -- 172.21.15.181:0/1548339540 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb424102690 msgr2=0x7fb424102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:52:14.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.125+0000 7fb42a4cd700 1 --2- 172.21.15.181:0/1548339540 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb424102690 0x7fb424102a70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb4140099c0 tx=0x7fb414009cd0 comp rx=0 tx=0).stop 2024-05-03T20:52:14.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.126+0000 7fb42a4cd700 1 -- 172.21.15.181:0/1548339540 shutdown_connections 2024-05-03T20:52:14.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.126+0000 7fb42a4cd700 1 --2- 172.21.15.181:0/1548339540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4241076e0 0x7fb42410f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:14.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.126+0000 7fb42a4cd700 1 --2- 172.21.15.181:0/1548339540 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb424102690 0x7fb424102a70 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:14.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.126+0000 7fb42a4cd700 1 --2- 172.21.15.181:0/1548339540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb424103040 0x7fb424107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:14.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.126+0000 7fb42a4cd700 1 -- 172.21.15.181:0/1548339540 >> 172.21.15.181:0/1548339540 conn(0x7fb4240783f0 msgr2=0x7fb424078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:52:14.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.126+0000 7fb42a4cd700 1 -- 172.21.15.181:0/1548339540 shutdown_connections 2024-05-03T20:52:14.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.126+0000 7fb42a4cd700 1 -- 172.21.15.181:0/1548339540 wait complete. 2024-05-03T20:52:14.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.126+0000 7fb42a4cd700 1 Processor -- start 2024-05-03T20:52:14.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.126+0000 7fb42a4cd700 1 -- start start 2024-05-03T20:52:14.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.127+0000 7fb42a4cd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb424102690 0x7fb42419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:52:14.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.127+0000 7fb42a4cd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb424103040 0x7fb42419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:52:14.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.127+0000 7fb42a4cd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4241076e0 0x7fb4241988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:52:14.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.127+0000 7fb42a4cd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb424112570 con 0x7fb424103040 2024-05-03T20:52:14.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.127+0000 7fb42a4cd700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb4241123f0 con 0x7fb424102690 2024-05-03T20:52:14.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.127+0000 7fb42a4cd700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb4241126f0 con 0x7fb4241076e0 2024-05-03T20:52:14.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.127+0000 7fb4237fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb424103040 0x7fb42419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:52:14.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.127+0000 7fb4237fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb424103040 0x7fb42419ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52048/0 (socket says 172.21.15.181:52048) 2024-05-03T20:52:14.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.127+0000 7fb4237fe700 1 -- 172.21.15.181:0/3650233183 learned_addr learned my addr 172.21.15.181:0/3650233183 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-03T20:52:14.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.127+0000 7fb428a6a700 1 --2- 172.21.15.181:0/3650233183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4241076e0 0x7fb4241988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:52:14.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.127+0000 7fb423fff700 1 --2- 172.21.15.181:0/3650233183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb424102690 0x7fb42419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:52:14.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.127+0000 7fb4237fe700 1 -- 172.21.15.181:0/3650233183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4241076e0 msgr2=0x7fb4241988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:52:14.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.127+0000 7fb4237fe700 1 --2- 172.21.15.181:0/3650233183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4241076e0 0x7fb4241988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:14.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.127+0000 7fb4237fe700 1 -- 172.21.15.181:0/3650233183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb424102690 msgr2=0x7fb42419e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:52:14.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.127+0000 7fb4237fe700 1 --2- 172.21.15.181:0/3650233183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb424102690 0x7fb42419e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:14.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.127+0000 7fb4237fe700 1 -- 172.21.15.181:0/3650233183 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb424199050 con 0x7fb424103040 2024-05-03T20:52:14.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.127+0000 7fb423fff700 1 --2- 172.21.15.181:0/3650233183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb424102690 0x7fb42419e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T20:52:14.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.128+0000 7fb4237fe700 1 --2- 172.21.15.181:0/3650233183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb424103040 0x7fb42419ec70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb41800e970 tx=0x7fb41800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:52:14.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.128+0000 7fb4217fa700 1 -- 172.21.15.181:0/3650233183 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41800cbd0 con 0x7fb424103040 2024-05-03T20:52:14.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.128+0000 7fb4217fa700 1 -- 172.21.15.181:0/3650233183 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb41800cd70 con 0x7fb424103040 2024-05-03T20:52:14.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.128+0000 7fb42a4cd700 1 -- 172.21.15.181:0/3650233183 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb424199340 con 0x7fb424103040 2024-05-03T20:52:14.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.128+0000 7fb4217fa700 1 -- 172.21.15.181:0/3650233183 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb418005680 con 0x7fb424103040 2024-05-03T20:52:14.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.128+0000 7fb42a4cd700 1 -- 172.21.15.181:0/3650233183 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4241a5600 con 0x7fb424103040 2024-05-03T20:52:14.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.129+0000 7fb42a4cd700 1 -- 172.21.15.181:0/3650233183 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb42406a190 con 0x7fb424103040 2024-05-03T20:52:14.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.130+0000 7fb4217fa700 1 -- 172.21.15.181:0/3650233183 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fb418004750 con 0x7fb424103040 2024-05-03T20:52:14.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.131+0000 7fb4217fa700 1 --2- 172.21.15.181:0/3650233183 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb40c0b2910 0x7fb40c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T20:52:14.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.131+0000 7fb423fff700 1 --2- 172.21.15.181:0/3650233183 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb40c0b2910 0x7fb40c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T20:52:14.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.131+0000 7fb4217fa700 1 -- 172.21.15.181:0/3650233183 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8095+0+0 (secure 0 0 0) 0x7fb4180e2490 con 0x7fb424103040 2024-05-03T20:52:14.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.131+0000 7fb423fff700 1 --2- 172.21.15.181:0/3650233183 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb40c0b2910 0x7fb40c0b4dd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb414000c00 tx=0x7fb414005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T20:52:14.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.135+0000 7fb4217fa700 1 -- 172.21.15.181:0/3650233183 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb4180aa190 con 0x7fb424103040 2024-05-03T20:52:14.336 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.337+0000 7fb42a4cd700 1 -- 172.21.15.181:0/3650233183 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb42402e510 con 0x7fb424103040 2024-05-03T20:52:14.713 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.714+0000 7fb4217fa700 1 -- 172.21.15.181:0/3650233183 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v86) ==== 169+0+0 (secure 0 0 0) 0x7fb4180a5020 con 0x7fb424103040 2024-05-03T20:52:14.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.717+0000 7fb42a4cd700 1 -- 172.21.15.181:0/3650233183 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb40c0b2910 msgr2=0x7fb40c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:52:14.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.717+0000 7fb42a4cd700 1 --2- 172.21.15.181:0/3650233183 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb40c0b2910 0x7fb40c0b4dd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb414000c00 tx=0x7fb414005940 comp rx=0 tx=0).stop 2024-05-03T20:52:14.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.717+0000 7fb42a4cd700 1 -- 172.21.15.181:0/3650233183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb424103040 msgr2=0x7fb42419ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T20:52:14.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.717+0000 7fb42a4cd700 1 --2- 172.21.15.181:0/3650233183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb424103040 0x7fb42419ec70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb41800e970 tx=0x7fb41800ec80 comp rx=0 tx=0).stop 2024-05-03T20:52:14.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.718+0000 7fb42a4cd700 1 -- 172.21.15.181:0/3650233183 shutdown_connections 2024-05-03T20:52:14.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.718+0000 7fb42a4cd700 1 --2- 172.21.15.181:0/3650233183 >> [v2:172.21.15.181:6800/1919423525,v1:172.21.15.181:6801/1919423525] conn(0x7fb40c0b2910 0x7fb40c0b4dd0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:14.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.718+0000 7fb42a4cd700 1 --2- 172.21.15.181:0/3650233183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4241076e0 0x7fb4241988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:14.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.718+0000 7fb42a4cd700 1 --2- 172.21.15.181:0/3650233183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb424102690 0x7fb42419e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:14.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.718+0000 7fb42a4cd700 1 --2- 172.21.15.181:0/3650233183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb424103040 0x7fb42419ec70 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T20:52:14.719 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.718+0000 7fb42a4cd700 1 -- 172.21.15.181:0/3650233183 >> 172.21.15.181:0/3650233183 conn(0x7fb4240783f0 msgr2=0x7fb424106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T20:52:14.719 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.718+0000 7fb42a4cd700 1 -- 172.21.15.181:0/3650233183 shutdown_connections 2024-05-03T20:52:14.719 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-03T20:52:14.718+0000 7fb42a4cd700 1 -- 172.21.15.181:0/3650233183 wait complete. 2024-05-03T20:52:14.719 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' removed 2024-05-03T20:52:14.962 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:14 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3650233183' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T20:52:15.106 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:14 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3650233183' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T20:52:15.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:14 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3650233183' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T20:52:15.431 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T20:52:15.458 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T20:52:15.491 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T20:52:15.525 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-03T20:52:15.526 DEBUG:teuthology.orchestra.run.smithi181:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-03T20:52:15.542 DEBUG:teuthology.orchestra.run.smithi196:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-03T20:52:15.559 DEBUG:teuthology.orchestra.run.smithi204:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-03T20:52:15.581 INFO:tasks.cephadm:Stopping all daemons... 2024-05-03T20:52:15.582 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-03T20:52:15.582 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mon.a 2024-05-03T20:52:15.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:15 smithi181 systemd[1]: Stopping Ceph mon.a for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:52:15.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:15 smithi196 ceph-mon[32858]: pgmap v845: 65 pgs: 65 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:52:15.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:15 smithi196 ceph-mon[32858]: from='client.? 172.21.15.181:0/3650233183' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T20:52:15.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:15 smithi196 ceph-mon[32858]: osdmap e86: 12 total, 12 up, 12 in 2024-05-03T20:52:15.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:15 smithi204 ceph-mon[33595]: pgmap v845: 65 pgs: 65 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:52:15.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:15 smithi204 ceph-mon[33595]: from='client.? 172.21.15.181:0/3650233183' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T20:52:15.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:15 smithi204 ceph-mon[33595]: osdmap e86: 12 total, 12 up, 12 in 2024-05-03T20:52:16.102 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:15 smithi181 ceph-mon[29801]: pgmap v845: 65 pgs: 65 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T20:52:16.102 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:15 smithi181 ceph-mon[29801]: from='client.? 172.21.15.181:0/3650233183' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T20:52:16.102 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:15 smithi181 ceph-mon[29801]: osdmap e86: 12 total, 12 up, 12 in 2024-05-03T20:52:16.102 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:15 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-a[29797]: 2024-05-03T20:52:15.878+0000 7f2810c6a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:52:16.103 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:15 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-a[29797]: 2024-05-03T20:52:15.878+0000 7f2810c6a700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-03T20:52:16.403 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:16 smithi181 podman[91175]: 2024-05-03 20:52:16.104390247 +0000 UTC m=+0.333054235 container died e1d7a382742ab709b40f3a0b5a0a07e7841de2ef94fa45bf0c0fb342d174427d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-03T20:52:16.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:16 smithi181 podman[91175]: 2024-05-03 20:52:16.513276361 +0000 UTC m=+0.741940357 container cleanup e1d7a382742ab709b40f3a0b5a0a07e7841de2ef94fa45bf0c0fb342d174427d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-05-03T20:52:16.712 INFO:journalctl@ceph.mon.a.smithi181.stdout:May 03 20:52:16 smithi181 bash[91175]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-a 2024-05-03T20:52:16.891 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mon.a.service' 2024-05-03T20:52:16.939 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:52:16.939 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-03T20:52:16.940 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-03T20:52:16.940 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mon.b 2024-05-03T20:52:17.248 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:16 smithi196 systemd[1]: Stopping Ceph mon.b for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:52:17.565 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:17 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-b[32854]: 2024-05-03T20:52:17.249+0000 7f049aa5e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:52:17.566 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:17 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-b[32854]: 2024-05-03T20:52:17.249+0000 7f049aa5e700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-03T20:52:17.566 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:17 smithi196 podman[68034]: 2024-05-03 20:52:17.342117257 +0000 UTC m=+0.200136689 container died d34180c69a8f5064d05d1b020a492c09eccab7f43bbfcdaf5fc7be169657d0fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-03T20:52:17.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:17 smithi196 podman[68034]: 2024-05-03 20:52:17.742490634 +0000 UTC m=+0.600510098 container cleanup d34180c69a8f5064d05d1b020a492c09eccab7f43bbfcdaf5fc7be169657d0fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T20:52:17.856 INFO:journalctl@ceph.mon.b.smithi196.stdout:May 03 20:52:17 smithi196 bash[68034]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-b 2024-05-03T20:52:18.085 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mon.b.service' 2024-05-03T20:52:18.134 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:52:18.134 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-03T20:52:18.134 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-03T20:52:18.134 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mon.c 2024-05-03T20:52:18.458 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:18 smithi204 systemd[1]: Stopping Ceph mon.c for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:52:18.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:18 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-c[33579]: 2024-05-03T20:52:18.459+0000 7fe02fff7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:52:18.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:18 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-c[33579]: 2024-05-03T20:52:18.459+0000 7fe02fff7700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-03T20:52:18.863 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:18 smithi204 podman[69054]: 2024-05-03 20:52:18.704716368 +0000 UTC m=+0.368798554 container died f777a02cfbdfa59207ff2857b228cbc86f34e9b2db2897c0c514e9f535d1ee45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-03T20:52:19.150 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:19 smithi204 podman[69054]: 2024-05-03 20:52:19.134834351 +0000 UTC m=+0.798916528 container cleanup f777a02cfbdfa59207ff2857b228cbc86f34e9b2db2897c0c514e9f535d1ee45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-c, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-03T20:52:19.150 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:19 smithi204 bash[69054]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-c 2024-05-03T20:52:19.558 INFO:journalctl@ceph.mon.c.smithi204.stdout:May 03 20:52:19 smithi204 podman[69066]: 2024-05-03 20:52:19.344972753 +0000 UTC m=+0.637454371 container remove f777a02cfbdfa59207ff2857b228cbc86f34e9b2db2897c0c514e9f535d1ee45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mon-c, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-03T20:52:19.572 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mon.c.service' 2024-05-03T20:52:19.620 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:52:19.620 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-03T20:52:19.621 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-03T20:52:19.621 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mgr.x 2024-05-03T20:52:19.950 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:52:19 smithi181 systemd[1]: Stopping Ceph mgr.x for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:52:20.212 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:52:19 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x[30033]: 2024-05-03T20:52:19.950+0000 7f4d57d8f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:52:20.212 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:52:19 smithi181 podman[91296]: 2024-05-03 20:52:19.982049858 +0000 UTC m=+0.166535559 container died 87c0517d69b19f49fd5b892b63c53789fc6a1a8eb5402683d1f28f956fc5805f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-03T20:52:20.609 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:52:20 smithi181 podman[91296]: 2024-05-03 20:52:20.376576404 +0000 UTC m=+0.561062099 container cleanup 87c0517d69b19f49fd5b892b63c53789fc6a1a8eb5402683d1f28f956fc5805f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-03T20:52:20.609 INFO:journalctl@ceph.mgr.x.smithi181.stdout:May 03 20:52:20 smithi181 bash[91296]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-x 2024-05-03T20:52:20.741 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mgr.x.service' 2024-05-03T20:52:20.774 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:52:20.775 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-03T20:52:20.775 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-03T20:52:20.775 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mgr.y 2024-05-03T20:52:21.082 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:52:20 smithi196 systemd[1]: Stopping Ceph mgr.y for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:52:21.356 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:52:21 smithi196 podman[68167]: 2024-05-03 20:52:21.083841719 +0000 UTC m=+0.119367304 container died fba3b2277bf8851913ed02d81741cb342fe4b0798c7ed1a5afc59ed11cadff47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-03T20:52:21.805 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:52:21 smithi196 podman[68167]: 2024-05-03 20:52:21.563473313 +0000 UTC m=+0.598998894 container cleanup fba3b2277bf8851913ed02d81741cb342fe4b0798c7ed1a5afc59ed11cadff47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-03T20:52:21.805 INFO:journalctl@ceph.mgr.y.smithi196.stdout:May 03 20:52:21 smithi196 bash[68167]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-y 2024-05-03T20:52:21.974 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mgr.y.service' 2024-05-03T20:52:22.010 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:52:22.010 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-03T20:52:22.010 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-03T20:52:22.010 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mgr.z 2024-05-03T20:52:22.333 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:52:22 smithi204 systemd[1]: Stopping Ceph mgr.z for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:52:22.612 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:52:22 smithi204 podman[69190]: 2024-05-03 20:52:22.335636515 +0000 UTC m=+0.127037206 container died a0992a0847c663ca457a72741f69a8ab88fa3277ef20e8aa5ea6d95980dfcbb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T20:52:22.964 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:52:22 smithi204 podman[69190]: 2024-05-03 20:52:22.714684406 +0000 UTC m=+0.506085108 container cleanup a0992a0847c663ca457a72741f69a8ab88fa3277ef20e8aa5ea6d95980dfcbb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS) 2024-05-03T20:52:22.964 INFO:journalctl@ceph.mgr.z.smithi204.stdout:May 03 20:52:22 smithi204 bash[69190]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mgr-z 2024-05-03T20:52:23.132 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mgr.z.service' 2024-05-03T20:52:23.167 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:52:23.167 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-03T20:52:23.167 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-03T20:52:23.168 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.0 2024-05-03T20:52:23.462 INFO:journalctl@ceph.osd.0.smithi181.stdout:May 03 20:52:23 smithi181 systemd[1]: Stopping Ceph osd.0 for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:52:23.962 INFO:journalctl@ceph.osd.0.smithi181.stdout:May 03 20:52:23 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-0[45040]: 2024-05-03T20:52:23.464+0000 7fbaf2ad8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:52:23.963 INFO:journalctl@ceph.osd.0.smithi181.stdout:May 03 20:52:23 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-0[45040]: 2024-05-03T20:52:23.464+0000 7fbaf2ad8700 -1 osd.0 86 *** Got signal Terminated *** 2024-05-03T20:52:23.963 INFO:journalctl@ceph.osd.0.smithi181.stdout:May 03 20:52:23 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-0[45040]: 2024-05-03T20:52:23.464+0000 7fbaf2ad8700 -1 osd.0 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T20:52:29.113 INFO:journalctl@ceph.osd.0.smithi181.stdout:May 03 20:52:28 smithi181 podman[91418]: 2024-05-03 20:52:28.849498698 +0000 UTC m=+5.492607060 container died b6ac617570975c2edfb92735b7f1453e8ffb95512f58c0713b57dcd6acb77d8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-0, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-03T20:52:29.468 INFO:journalctl@ceph.osd.0.smithi181.stdout:May 03 20:52:29 smithi181 podman[91418]: 2024-05-03 20:52:29.213784927 +0000 UTC m=+5.856893295 container cleanup b6ac617570975c2edfb92735b7f1453e8ffb95512f58c0713b57dcd6acb77d8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-03T20:52:29.468 INFO:journalctl@ceph.osd.0.smithi181.stdout:May 03 20:52:29 smithi181 bash[91418]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-0 2024-05-03T20:52:29.824 INFO:journalctl@ceph.osd.0.smithi181.stdout:May 03 20:52:29 smithi181 podman[91435]: 2024-05-03 20:52:29.490092161 +0000 UTC m=+0.639236700 container remove b6ac617570975c2edfb92735b7f1453e8ffb95512f58c0713b57dcd6acb77d8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True) 2024-05-03T20:52:30.142 INFO:journalctl@ceph.osd.0.smithi181.stdout:May 03 20:52:29 smithi181 podman[91525]: 2024-05-03 20:52:29.728605706 +0000 UTC m=+0.018200591 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T20:52:30.142 INFO:journalctl@ceph.osd.0.smithi181.stdout:May 03 20:52:29 smithi181 podman[91525]: 2024-05-03 20:52:29.83268917 +0000 UTC m=+0.122284054 container create 5e2d1f37d986939b2ce46281e5338ae3f6c73e649dd4906d927875c984c29090 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-03T20:52:30.143 INFO:journalctl@ceph.osd.0.smithi181.stdout:May 03 20:52:30 smithi181 podman[91525]: 2024-05-03 20:52:30.051950484 +0000 UTC m=+0.341545369 container init 5e2d1f37d986939b2ce46281e5338ae3f6c73e649dd4906d927875c984c29090 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240430, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-03T20:52:30.143 INFO:journalctl@ceph.osd.0.smithi181.stdout:May 03 20:52:30 smithi181 podman[91525]: 2024-05-03 20:52:30.055198788 +0000 UTC m=+0.344793679 container start 5e2d1f37d986939b2ce46281e5338ae3f6c73e649dd4906d927875c984c29090 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T20:52:30.462 INFO:journalctl@ceph.osd.0.smithi181.stdout:May 03 20:52:30 smithi181 podman[91525]: 2024-05-03 20:52:30.143673856 +0000 UTC m=+0.433268764 container attach 5e2d1f37d986939b2ce46281e5338ae3f6c73e649dd4906d927875c984c29090 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.build-date=20240430) 2024-05-03T20:52:30.463 INFO:journalctl@ceph.osd.0.smithi181.stdout:May 03 20:52:30 smithi181 podman[91525]: 2024-05-03 20:52:30.358209804 +0000 UTC m=+0.647804699 container died 5e2d1f37d986939b2ce46281e5338ae3f6c73e649dd4906d927875c984c29090 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-03T20:52:31.030 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.0.service' 2024-05-03T20:52:31.070 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:52:31.071 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-03T20:52:31.071 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-03T20:52:31.071 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.1 2024-05-03T20:52:31.349 INFO:journalctl@ceph.osd.1.smithi196.stdout:May 03 20:52:31 smithi196 systemd[1]: Stopping Ceph osd.1 for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:52:31.606 INFO:journalctl@ceph.osd.1.smithi196.stdout:May 03 20:52:31 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-1[37481]: 2024-05-03T20:52:31.350+0000 7ff07aa67700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:52:31.606 INFO:journalctl@ceph.osd.1.smithi196.stdout:May 03 20:52:31 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-1[37481]: 2024-05-03T20:52:31.350+0000 7ff07aa67700 -1 osd.1 86 *** Got signal Terminated *** 2024-05-03T20:52:31.606 INFO:journalctl@ceph.osd.1.smithi196.stdout:May 03 20:52:31 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-1[37481]: 2024-05-03T20:52:31.350+0000 7ff07aa67700 -1 osd.1 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T20:52:36.998 INFO:journalctl@ceph.osd.1.smithi196.stdout:May 03 20:52:36 smithi196 podman[68296]: 2024-05-03 20:52:36.651046535 +0000 UTC m=+5.385162929 container died f7b280b00d60b5c9cd4cd69b438be2614a8ea07cfd23a759779307a45e8657e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-1, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-03T20:52:37.356 INFO:journalctl@ceph.osd.1.smithi196.stdout:May 03 20:52:37 smithi196 podman[68296]: 2024-05-03 20:52:37.000376546 +0000 UTC m=+5.734492943 container cleanup f7b280b00d60b5c9cd4cd69b438be2614a8ea07cfd23a759779307a45e8657e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-03T20:52:37.356 INFO:journalctl@ceph.osd.1.smithi196.stdout:May 03 20:52:37 smithi196 bash[68296]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-1 2024-05-03T20:52:37.356 INFO:journalctl@ceph.osd.1.smithi196.stdout:May 03 20:52:37 smithi196 podman[68314]: 2024-05-03 20:52:37.218072368 +0000 UTC m=+0.561103163 container remove f7b280b00d60b5c9cd4cd69b438be2614a8ea07cfd23a759779307a45e8657e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-1, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T20:52:37.919 INFO:journalctl@ceph.osd.1.smithi196.stdout:May 03 20:52:37 smithi196 podman[68407]: 2024-05-03 20:52:37.524116673 +0000 UTC m=+0.017931799 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T20:52:37.920 INFO:journalctl@ceph.osd.1.smithi196.stdout:May 03 20:52:37 smithi196 podman[68407]: 2024-05-03 20:52:37.643987231 +0000 UTC m=+0.137802385 container create 5b5dd56eec1a97b582564b911fbf58a9792c4e05746c034e1a7eb47f1c4ef097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430) 2024-05-03T20:52:37.920 INFO:journalctl@ceph.osd.1.smithi196.stdout:May 03 20:52:37 smithi196 podman[68407]: 2024-05-03 20:52:37.8630724 +0000 UTC m=+0.356887512 container init 5b5dd56eec1a97b582564b911fbf58a9792c4e05746c034e1a7eb47f1c4ef097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-05-03T20:52:37.920 INFO:journalctl@ceph.osd.1.smithi196.stdout:May 03 20:52:37 smithi196 podman[68407]: 2024-05-03 20:52:37.866273335 +0000 UTC m=+0.360088448 container start 5b5dd56eec1a97b582564b911fbf58a9792c4e05746c034e1a7eb47f1c4ef097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-03T20:52:37.920 INFO:journalctl@ceph.osd.1.smithi196.stdout:May 03 20:52:37 smithi196 podman[68407]: 2024-05-03 20:52:37.92130246 +0000 UTC m=+0.415117591 container attach 5b5dd56eec1a97b582564b911fbf58a9792c4e05746c034e1a7eb47f1c4ef097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2) 2024-05-03T20:52:38.180 INFO:journalctl@ceph.osd.1.smithi196.stdout:May 03 20:52:38 smithi196 podman[68407]: 2024-05-03 20:52:38.154686152 +0000 UTC m=+0.648501274 container died 5b5dd56eec1a97b582564b911fbf58a9792c4e05746c034e1a7eb47f1c4ef097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-03T20:52:38.718 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.1.service' 2024-05-03T20:52:38.755 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:52:38.755 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-03T20:52:38.755 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-03T20:52:38.756 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.2 2024-05-03T20:52:39.067 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 03 20:52:38 smithi204 systemd[1]: Stopping Ceph osd.2 for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:52:39.363 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 03 20:52:39 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-2[38742]: 2024-05-03T20:52:39.068+0000 7efc2718b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:52:39.363 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 03 20:52:39 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-2[38742]: 2024-05-03T20:52:39.068+0000 7efc2718b700 -1 osd.2 86 *** Got signal Terminated *** 2024-05-03T20:52:39.363 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 03 20:52:39 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-2[38742]: 2024-05-03T20:52:39.068+0000 7efc2718b700 -1 osd.2 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T20:52:44.692 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 03 20:52:44 smithi204 podman[69314]: 2024-05-03 20:52:44.411186535 +0000 UTC m=+5.463461261 container died 36dd222b958639aab8d679d6aa624232ad0867a03e29e5ca8400ae67ba0268ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T20:52:44.952 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 03 20:52:44 smithi204 podman[69314]: 2024-05-03 20:52:44.693830798 +0000 UTC m=+5.746105532 container cleanup 36dd222b958639aab8d679d6aa624232ad0867a03e29e5ca8400ae67ba0268ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T20:52:44.952 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 03 20:52:44 smithi204 bash[69314]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-2 2024-05-03T20:52:45.314 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 03 20:52:44 smithi204 podman[69332]: 2024-05-03 20:52:44.953886923 +0000 UTC m=+0.539260960 container remove 36dd222b958639aab8d679d6aa624232ad0867a03e29e5ca8400ae67ba0268ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-05-03T20:52:45.565 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 03 20:52:45 smithi204 podman[69419]: 2024-05-03 20:52:45.218993656 +0000 UTC m=+0.018381802 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T20:52:45.565 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 03 20:52:45 smithi204 podman[69419]: 2024-05-03 20:52:45.3473851 +0000 UTC m=+0.146773288 container create afc846ad5045080a826595f894a1941a2e339ce9d1a306263170b8ab13013e4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS) 2024-05-03T20:52:45.565 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 03 20:52:45 smithi204 podman[69419]: 2024-05-03 20:52:45.566730482 +0000 UTC m=+0.366118628 container init afc846ad5045080a826595f894a1941a2e339ce9d1a306263170b8ab13013e4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-2-deactivate, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-03T20:52:45.863 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 03 20:52:45 smithi204 podman[69419]: 2024-05-03 20:52:45.570134102 +0000 UTC m=+0.369522254 container start afc846ad5045080a826595f894a1941a2e339ce9d1a306263170b8ab13013e4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T20:52:45.863 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 03 20:52:45 smithi204 podman[69419]: 2024-05-03 20:52:45.675062862 +0000 UTC m=+0.474451009 container attach afc846ad5045080a826595f894a1941a2e339ce9d1a306263170b8ab13013e4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-05-03T20:52:46.159 INFO:journalctl@ceph.osd.2.smithi204.stdout:May 03 20:52:45 smithi204 podman[69419]: 2024-05-03 20:52:45.869033402 +0000 UTC m=+0.668421551 container died afc846ad5045080a826595f894a1941a2e339ce9d1a306263170b8ab13013e4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-03T20:52:46.436 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.2.service' 2024-05-03T20:52:46.474 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:52:46.475 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-03T20:52:46.475 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-03T20:52:46.475 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.3 2024-05-03T20:52:46.765 INFO:journalctl@ceph.osd.3.smithi181.stdout:May 03 20:52:46 smithi181 systemd[1]: Stopping Ceph osd.3 for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:52:47.212 INFO:journalctl@ceph.osd.3.smithi181.stdout:May 03 20:52:46 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-3[51875]: 2024-05-03T20:52:46.766+0000 7fc4e260d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:52:47.213 INFO:journalctl@ceph.osd.3.smithi181.stdout:May 03 20:52:46 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-3[51875]: 2024-05-03T20:52:46.766+0000 7fc4e260d700 -1 osd.3 86 *** Got signal Terminated *** 2024-05-03T20:52:47.213 INFO:journalctl@ceph.osd.3.smithi181.stdout:May 03 20:52:46 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-3[51875]: 2024-05-03T20:52:46.766+0000 7fc4e260d700 -1 osd.3 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T20:52:50.463 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:52:50 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:52:50.203+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.734848+0000 front 2024-05-03T20:52:27.734877+0000 (oldest deadline 2024-05-03T20:52:50.035068+0000) 2024-05-03T20:52:50.613 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:52:50 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:52:50.188+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:52:51.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:52:50 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:52:50.792+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:52:51.212 INFO:journalctl@ceph.osd.3.smithi181.stdout:May 03 20:52:50 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-3[51875]: 2024-05-03T20:52:50.836+0000 7fc4dabf9700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.184880+0000 front 2024-05-03T20:52:27.184910+0000 (oldest deadline 2024-05-03T20:52:50.685041+0000) 2024-05-03T20:52:51.380 INFO:journalctl@ceph.osd.4.smithi196.stdout:May 03 20:52:51 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-4[44447]: 2024-05-03T20:52:51.300+0000 7fd0aff8f700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.588229+0000 front 2024-05-03T20:52:25.588175+0000 (oldest deadline 2024-05-03T20:52:50.888233+0000) 2024-05-03T20:52:51.613 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:52:51 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:52:51.197+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:52:51.613 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:52:51 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:52:51.424+0000 7f48bb9b5700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.159238+0000 front 2024-05-03T20:52:27.159389+0000 (oldest deadline 2024-05-03T20:52:50.659055+0000) 2024-05-03T20:52:51.712 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:52:51 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:52:51.241+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.734848+0000 front 2024-05-03T20:52:27.734877+0000 (oldest deadline 2024-05-03T20:52:50.035068+0000) 2024-05-03T20:52:52.106 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:52:51 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:52:51.813+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:52:52.532 INFO:journalctl@ceph.osd.3.smithi181.stdout:May 03 20:52:52 smithi181 podman[91658]: 2024-05-03 20:52:52.274129732 +0000 UTC m=+5.603645026 container died dd1a437cd7b258663e90a2d7dcabe8c37fff6572bc10d99e0766c89fa7b1e5d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-3, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-03T20:52:52.533 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:52:52 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:52:52.229+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.734848+0000 front 2024-05-03T20:52:27.734877+0000 (oldest deadline 2024-05-03T20:52:50.035068+0000) 2024-05-03T20:52:52.606 INFO:journalctl@ceph.osd.4.smithi196.stdout:May 03 20:52:52 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-4[44447]: 2024-05-03T20:52:52.311+0000 7fd0aff8f700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.588229+0000 front 2024-05-03T20:52:25.588175+0000 (oldest deadline 2024-05-03T20:52:50.888233+0000) 2024-05-03T20:52:52.613 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:52:52 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:52:52.410+0000 7f48bb9b5700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.159238+0000 front 2024-05-03T20:52:27.159389+0000 (oldest deadline 2024-05-03T20:52:50.659055+0000) 2024-05-03T20:52:52.613 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:52:52 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:52:52.163+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:52:52.890 INFO:journalctl@ceph.osd.3.smithi181.stdout:May 03 20:52:52 smithi181 podman[91658]: 2024-05-03 20:52:52.633053605 +0000 UTC m=+5.962568906 container cleanup dd1a437cd7b258663e90a2d7dcabe8c37fff6572bc10d99e0766c89fa7b1e5d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-3, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240430) 2024-05-03T20:52:52.890 INFO:journalctl@ceph.osd.3.smithi181.stdout:May 03 20:52:52 smithi181 bash[91658]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-3 2024-05-03T20:52:53.106 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:52:52 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:52:52.851+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:52:53.213 INFO:journalctl@ceph.osd.3.smithi181.stdout:May 03 20:52:52 smithi181 podman[91675]: 2024-05-03 20:52:52.892503298 +0000 UTC m=+0.616848204 container remove dd1a437cd7b258663e90a2d7dcabe8c37fff6572bc10d99e0766c89fa7b1e5d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-03T20:52:53.541 INFO:journalctl@ceph.osd.3.smithi181.stdout:May 03 20:52:53 smithi181 podman[91767]: 2024-05-03 20:52:53.251808323 +0000 UTC m=+0.105136394 container create 3bbc1c0e6de8290bf8c0025eabe2ba4a420cd2d12d1509644b92b917333691b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430) 2024-05-03T20:52:53.541 INFO:journalctl@ceph.osd.3.smithi181.stdout:May 03 20:52:53 smithi181 podman[91767]: 2024-05-03 20:52:53.164631068 +0000 UTC m=+0.017959145 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T20:52:53.541 INFO:journalctl@ceph.osd.3.smithi181.stdout:May 03 20:52:53 smithi181 podman[91767]: 2024-05-03 20:52:53.404065008 +0000 UTC m=+0.257393078 container init 3bbc1c0e6de8290bf8c0025eabe2ba4a420cd2d12d1509644b92b917333691b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-03T20:52:53.542 INFO:journalctl@ceph.osd.3.smithi181.stdout:May 03 20:52:53 smithi181 podman[91767]: 2024-05-03 20:52:53.40730665 +0000 UTC m=+0.260634716 container start 3bbc1c0e6de8290bf8c0025eabe2ba4a420cd2d12d1509644b92b917333691b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-03T20:52:53.542 INFO:journalctl@ceph.osd.3.smithi181.stdout:May 03 20:52:53 smithi181 podman[91767]: 2024-05-03 20:52:53.487551672 +0000 UTC m=+0.340879737 container attach 3bbc1c0e6de8290bf8c0025eabe2ba4a420cd2d12d1509644b92b917333691b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-03T20:52:53.542 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:52:53 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:52:53.239+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.734848+0000 front 2024-05-03T20:52:27.734877+0000 (oldest deadline 2024-05-03T20:52:50.035068+0000) 2024-05-03T20:52:53.542 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:52:53 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:52:53.317+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:52:53.606 INFO:journalctl@ceph.osd.4.smithi196.stdout:May 03 20:52:53 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-4[44447]: 2024-05-03T20:52:53.318+0000 7fd0aff8f700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.588229+0000 front 2024-05-03T20:52:25.588175+0000 (oldest deadline 2024-05-03T20:52:50.888233+0000) 2024-05-03T20:52:53.613 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:52:53 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:52:53.449+0000 7f48bb9b5700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.159238+0000 front 2024-05-03T20:52:27.159389+0000 (oldest deadline 2024-05-03T20:52:50.659055+0000) 2024-05-03T20:52:53.613 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:52:53 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:52:53.211+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:52:53.962 INFO:journalctl@ceph.osd.3.smithi181.stdout:May 03 20:52:53 smithi181 podman[91767]: 2024-05-03 20:52:53.709695031 +0000 UTC m=+0.563023114 container died 3bbc1c0e6de8290bf8c0025eabe2ba4a420cd2d12d1509644b92b917333691b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-3-deactivate, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-03T20:52:54.106 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:52:53 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:52:53.853+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:52:54.323 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:52:54 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:52:54.203+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.734848+0000 front 2024-05-03T20:52:27.734877+0000 (oldest deadline 2024-05-03T20:52:50.035068+0000) 2024-05-03T20:52:54.348 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.3.service' 2024-05-03T20:52:54.387 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:52:54.387 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-03T20:52:54.387 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-03T20:52:54.388 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.4 2024-05-03T20:52:54.489 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:52:54 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:52:54.164+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:52:54.606 INFO:journalctl@ceph.osd.4.smithi196.stdout:May 03 20:52:54 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-4[44447]: 2024-05-03T20:52:54.292+0000 7fd0aff8f700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.588229+0000 front 2024-05-03T20:52:25.588175+0000 (oldest deadline 2024-05-03T20:52:50.888233+0000) 2024-05-03T20:52:54.606 INFO:journalctl@ceph.osd.4.smithi196.stdout:May 03 20:52:54 smithi196 systemd[1]: Stopping Ceph osd.4 for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:52:54.712 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:52:54 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:52:54.342+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:52:54.863 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:52:54 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:52:54.490+0000 7f48bb9b5700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.159238+0000 front 2024-05-03T20:52:27.159389+0000 (oldest deadline 2024-05-03T20:52:50.659055+0000) 2024-05-03T20:52:55.106 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:52:54 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:52:54.817+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:52:55.106 INFO:journalctl@ceph.osd.4.smithi196.stdout:May 03 20:52:54 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-4[44447]: 2024-05-03T20:52:54.718+0000 7fd0b399b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:52:55.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:May 03 20:52:54 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-4[44447]: 2024-05-03T20:52:54.718+0000 7fd0b399b700 -1 osd.4 86 *** Got signal Terminated *** 2024-05-03T20:52:55.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:May 03 20:52:54 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-4[44447]: 2024-05-03T20:52:54.718+0000 7fd0b399b700 -1 osd.4 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T20:52:55.448 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:52:55 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:52:55.178+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:52:55.462 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:52:55 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:52:55.175+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.734848+0000 front 2024-05-03T20:52:27.734877+0000 (oldest deadline 2024-05-03T20:52:50.035068+0000) 2024-05-03T20:52:55.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:52:55 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:52:55.364+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:52:55.606 INFO:journalctl@ceph.osd.4.smithi196.stdout:May 03 20:52:55 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-4[44447]: 2024-05-03T20:52:55.287+0000 7fd0aff8f700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.588229+0000 front 2024-05-03T20:52:25.588175+0000 (oldest deadline 2024-05-03T20:52:50.888233+0000) 2024-05-03T20:52:55.863 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:52:55 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:52:55.449+0000 7f48bb9b5700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.159238+0000 front 2024-05-03T20:52:27.159389+0000 (oldest deadline 2024-05-03T20:52:50.659055+0000) 2024-05-03T20:52:56.106 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:52:55 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:52:55.769+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:52:56.435 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:52:56 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:52:56.137+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:52:56.462 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:52:56 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:52:56.212+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.734848+0000 front 2024-05-03T20:52:27.734877+0000 (oldest deadline 2024-05-03T20:52:50.035068+0000) 2024-05-03T20:52:56.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:52:56 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:52:56.414+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:52:56.606 INFO:journalctl@ceph.osd.4.smithi196.stdout:May 03 20:52:56 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-4[44447]: 2024-05-03T20:52:56.252+0000 7fd0aff8f700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.588229+0000 front 2024-05-03T20:52:25.588175+0000 (oldest deadline 2024-05-03T20:52:50.888233+0000) 2024-05-03T20:52:56.863 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:52:56 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:52:56.436+0000 7f48bb9b5700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.159238+0000 front 2024-05-03T20:52:27.159389+0000 (oldest deadline 2024-05-03T20:52:50.659055+0000) 2024-05-03T20:52:57.106 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:52:56 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:52:56.735+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:52:57.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:52:57 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:52:57.109+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:52:57.462 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:52:57 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:52:57.417+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:52:57.463 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:52:57 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:52:57.181+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.734848+0000 front 2024-05-03T20:52:27.734877+0000 (oldest deadline 2024-05-03T20:52:50.035068+0000) 2024-05-03T20:52:57.606 INFO:journalctl@ceph.osd.4.smithi196.stdout:May 03 20:52:57 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-4[44447]: 2024-05-03T20:52:57.225+0000 7fd0aff8f700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.588229+0000 front 2024-05-03T20:52:25.588175+0000 (oldest deadline 2024-05-03T20:52:50.888233+0000) 2024-05-03T20:52:57.863 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:52:57 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:52:57.411+0000 7f48bb9b5700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.159238+0000 front 2024-05-03T20:52:27.159389+0000 (oldest deadline 2024-05-03T20:52:50.659055+0000) 2024-05-03T20:52:58.106 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:52:57 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:52:57.871+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.627053+0000 front 2024-05-03T20:52:33.627200+0000 (oldest deadline 2024-05-03T20:52:57.127246+0000) 2024-05-03T20:52:58.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:52:57 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:52:57.785+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:52:58.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:52:57 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:52:57.785+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:52:58.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:52:58 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:52:58.109+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:52:58.454 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:52:58 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:52:58.185+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.734848+0000 front 2024-05-03T20:52:27.734877+0000 (oldest deadline 2024-05-03T20:52:50.035068+0000) 2024-05-03T20:52:58.606 INFO:journalctl@ceph.osd.4.smithi196.stdout:May 03 20:52:58 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-4[44447]: 2024-05-03T20:52:58.209+0000 7fd0aff8f700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.588229+0000 front 2024-05-03T20:52:25.588175+0000 (oldest deadline 2024-05-03T20:52:50.888233+0000) 2024-05-03T20:52:58.712 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:52:58 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:52:58.455+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:52:58.863 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:52:58 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:52:58.379+0000 7f48bb9b5700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.159238+0000 front 2024-05-03T20:52:27.159389+0000 (oldest deadline 2024-05-03T20:52:50.659055+0000) 2024-05-03T20:52:59.106 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:52:58 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:52:58.822+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.627053+0000 front 2024-05-03T20:52:33.627200+0000 (oldest deadline 2024-05-03T20:52:57.127246+0000) 2024-05-03T20:52:59.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:52:58 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:52:58.827+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:52:59.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:52:58 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:52:58.827+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:52:59.396 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:52:59 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:52:59.116+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:52:59.462 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:52:59 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:52:59.163+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.734848+0000 front 2024-05-03T20:52:27.734877+0000 (oldest deadline 2024-05-03T20:52:50.035068+0000) 2024-05-03T20:52:59.606 INFO:journalctl@ceph.osd.4.smithi196.stdout:May 03 20:52:59 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-4[44447]: 2024-05-03T20:52:59.231+0000 7fd0aff8f700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.588229+0000 front 2024-05-03T20:52:25.588175+0000 (oldest deadline 2024-05-03T20:52:50.888233+0000) 2024-05-03T20:52:59.863 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:52:59 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:52:59.397+0000 7f48bb9b5700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.159238+0000 front 2024-05-03T20:52:27.159389+0000 (oldest deadline 2024-05-03T20:52:50.659055+0000) 2024-05-03T20:52:59.962 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:52:59 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:52:59.494+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:52:59.963 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:52:59 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:52:59.494+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:00.106 INFO:journalctl@ceph.osd.4.smithi196.stdout:May 03 20:53:00 smithi196 podman[68538]: 2024-05-03 20:53:00.069152874 +0000 UTC m=+5.483456115 container died b193e77ecca835b3888b5cd66116959de62f86d040a9160bfd590b41ca2a1930 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-03T20:53:00.107 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:52:59 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:52:59.819+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.627053+0000 front 2024-05-03T20:52:33.627200+0000 (oldest deadline 2024-05-03T20:52:57.127246+0000) 2024-05-03T20:53:00.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:52:59 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:52:59.863+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:00.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:52:59 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:52:59.863+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:00.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:00 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:52:59.999+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:00.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:00 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:00.135+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:00.460 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:00 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:00.209+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.734848+0000 front 2024-05-03T20:52:27.734877+0000 (oldest deadline 2024-05-03T20:52:50.035068+0000) 2024-05-03T20:53:00.461 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:00 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:00.209+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:35.836984+0000 front 2024-05-03T20:52:35.837125+0000 (oldest deadline 2024-05-03T20:52:59.337050+0000) 2024-05-03T20:53:00.606 INFO:journalctl@ceph.osd.4.smithi196.stdout:May 03 20:53:00 smithi196 podman[68538]: 2024-05-03 20:53:00.359892184 +0000 UTC m=+5.774195417 container cleanup b193e77ecca835b3888b5cd66116959de62f86d040a9160bfd590b41ca2a1930 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-4, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T20:53:00.606 INFO:journalctl@ceph.osd.4.smithi196.stdout:May 03 20:53:00 smithi196 bash[68538]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-4 2024-05-03T20:53:00.606 INFO:journalctl@ceph.osd.4.smithi196.stdout:May 03 20:53:00 smithi196 podman[68556]: 2024-05-03 20:53:00.49417068 +0000 UTC m=+0.421334598 container remove b193e77ecca835b3888b5cd66116959de62f86d040a9160bfd590b41ca2a1930 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T20:53:00.712 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:00 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:00.461+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:00.712 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:00 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:00.461+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:00.863 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:00 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:53:00.416+0000 7f48bb9b5700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.159238+0000 front 2024-05-03T20:52:27.159389+0000 (oldest deadline 2024-05-03T20:52:50.659055+0000) 2024-05-03T20:53:01.106 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:00 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:00.850+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.627053+0000 front 2024-05-03T20:52:33.627200+0000 (oldest deadline 2024-05-03T20:52:57.127246+0000) 2024-05-03T20:53:01.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:May 03 20:53:00 smithi196 podman[68647]: 2024-05-03 20:53:00.886690364 +0000 UTC m=+0.110947015 container create 517721cb972a61123822017e2f465cea8f6ce06c32f4901b4368d822dfc1c8df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430) 2024-05-03T20:53:01.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:May 03 20:53:00 smithi196 podman[68647]: 2024-05-03 20:53:00.79337736 +0000 UTC m=+0.017633978 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T20:53:01.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:00 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:00.871+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:01.108 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:00 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:00.871+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:01.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:00 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:00.979+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:01.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:01 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:01.146+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:01.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:01 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:01.146+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:01.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:May 03 20:53:01 smithi196 podman[68647]: 2024-05-03 20:53:01.13048421 +0000 UTC m=+0.354740815 container init 517721cb972a61123822017e2f465cea8f6ce06c32f4901b4368d822dfc1c8df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-03T20:53:01.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:May 03 20:53:01 smithi196 podman[68647]: 2024-05-03 20:53:01.13392365 +0000 UTC m=+0.358180264 container start 517721cb972a61123822017e2f465cea8f6ce06c32f4901b4368d822dfc1c8df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-03T20:53:01.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:May 03 20:53:01 smithi196 podman[68647]: 2024-05-03 20:53:01.213851461 +0000 UTC m=+0.438108068 container attach 517721cb972a61123822017e2f465cea8f6ce06c32f4901b4368d822dfc1c8df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-03T20:53:01.462 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:01 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:01.426+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:01.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:01 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:01.426+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:01.463 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:01 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:01.177+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.734848+0000 front 2024-05-03T20:52:27.734877+0000 (oldest deadline 2024-05-03T20:52:50.035068+0000) 2024-05-03T20:53:01.463 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:01 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:01.177+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:35.836984+0000 front 2024-05-03T20:52:35.837125+0000 (oldest deadline 2024-05-03T20:52:59.337050+0000) 2024-05-03T20:53:01.761 INFO:journalctl@ceph.osd.4.smithi196.stdout:May 03 20:53:01 smithi196 podman[68647]: 2024-05-03 20:53:01.429895326 +0000 UTC m=+0.654151940 container died 517721cb972a61123822017e2f465cea8f6ce06c32f4901b4368d822dfc1c8df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-03T20:53:01.863 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:01 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:53:01.374+0000 7f48bb9b5700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.159238+0000 front 2024-05-03T20:52:27.159389+0000 (oldest deadline 2024-05-03T20:52:50.659055+0000) 2024-05-03T20:53:01.863 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:01 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:53:01.374+0000 7f48bb9b5700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:35.960174+0000 front 2024-05-03T20:52:35.960342+0000 (oldest deadline 2024-05-03T20:53:01.260244+0000) 2024-05-03T20:53:02.017 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:01 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:01.838+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.627053+0000 front 2024-05-03T20:52:33.627200+0000 (oldest deadline 2024-05-03T20:52:57.127246+0000) 2024-05-03T20:53:02.018 INFO:journalctl@ceph.osd.4.smithi196.stdout:May 03 20:53:02 smithi196 podman[68674]: 2024-05-03 20:53:02.000816598 +0000 UTC m=+0.562496003 container remove 517721cb972a61123822017e2f465cea8f6ce06c32f4901b4368d822dfc1c8df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-03T20:53:02.018 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:01 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:01.919+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:02.018 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:01 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:01.920+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:02.022 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.4.service' 2024-05-03T20:53:02.059 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:53:02.059 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-03T20:53:02.059 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-03T20:53:02.059 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.5 2024-05-03T20:53:02.190 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:02 smithi204 systemd[1]: Stopping Ceph osd.5 for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:53:02.191 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:01 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:01.937+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:02.462 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:02 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:02.403+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:02.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:02 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:02.403+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:02.463 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:02 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:02.159+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.734848+0000 front 2024-05-03T20:52:27.734877+0000 (oldest deadline 2024-05-03T20:52:50.035068+0000) 2024-05-03T20:53:02.463 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:02 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:02.159+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:35.836984+0000 front 2024-05-03T20:52:35.837125+0000 (oldest deadline 2024-05-03T20:52:59.337050+0000) 2024-05-03T20:53:02.613 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:02 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:02.191+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:02.613 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:02 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:02.191+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:02.613 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:02 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:53:02.364+0000 7f48bb9b5700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.159238+0000 front 2024-05-03T20:52:27.159389+0000 (oldest deadline 2024-05-03T20:52:50.659055+0000) 2024-05-03T20:53:02.614 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:02 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:53:02.364+0000 7f48bb9b5700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:35.960174+0000 front 2024-05-03T20:52:35.960342+0000 (oldest deadline 2024-05-03T20:53:01.260244+0000) 2024-05-03T20:53:02.614 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:02 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:53:02.372+0000 7f48bf3c1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:53:02.614 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:02 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:53:02.372+0000 7f48bf3c1700 -1 osd.5 86 *** Got signal Terminated *** 2024-05-03T20:53:02.614 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:02 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:53:02.372+0000 7f48bf3c1700 -1 osd.5 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T20:53:03.106 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:02 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:02.820+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.627053+0000 front 2024-05-03T20:52:33.627200+0000 (oldest deadline 2024-05-03T20:52:57.127246+0000) 2024-05-03T20:53:03.106 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:02 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:02.942+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:03.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:02 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:02.942+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:03.229 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:02 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:02.891+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:03.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:03 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:03.436+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:03.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:03 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:03.436+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:03.463 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:03 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:03.206+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.734848+0000 front 2024-05-03T20:52:27.734877+0000 (oldest deadline 2024-05-03T20:52:50.035068+0000) 2024-05-03T20:53:03.463 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:03 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:03.206+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:35.836984+0000 front 2024-05-03T20:52:35.837125+0000 (oldest deadline 2024-05-03T20:52:59.337050+0000) 2024-05-03T20:53:03.613 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:03 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:53:03.388+0000 7f48bb9b5700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.159238+0000 front 2024-05-03T20:52:27.159389+0000 (oldest deadline 2024-05-03T20:52:50.659055+0000) 2024-05-03T20:53:03.613 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:03 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:53:03.388+0000 7f48bb9b5700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:35.960174+0000 front 2024-05-03T20:52:35.960342+0000 (oldest deadline 2024-05-03T20:53:01.260244+0000) 2024-05-03T20:53:03.613 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:03 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:03.230+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:03.613 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:03 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:03.230+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:04.106 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:03 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:03.792+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.627053+0000 front 2024-05-03T20:52:33.627200+0000 (oldest deadline 2024-05-03T20:52:57.127246+0000) 2024-05-03T20:53:04.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:03 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:03.919+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:04.108 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:03 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:03.919+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:04.196 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:03 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:03.892+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:04.613 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:04 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:53:04.388+0000 7f48bb9b5700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.159238+0000 front 2024-05-03T20:52:27.159389+0000 (oldest deadline 2024-05-03T20:52:50.659055+0000) 2024-05-03T20:53:04.613 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:04 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:53:04.388+0000 7f48bb9b5700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:35.960174+0000 front 2024-05-03T20:52:35.960342+0000 (oldest deadline 2024-05-03T20:53:01.260244+0000) 2024-05-03T20:53:04.614 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:04 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:04.197+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:04.614 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:04 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:04.197+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:04.712 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:04 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:04.447+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:04.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:04 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:04.447+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:04.713 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:04 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:04.226+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.734848+0000 front 2024-05-03T20:52:27.734877+0000 (oldest deadline 2024-05-03T20:52:50.035068+0000) 2024-05-03T20:53:04.713 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:04 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:04.226+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:35.836984+0000 front 2024-05-03T20:52:35.837125+0000 (oldest deadline 2024-05-03T20:52:59.337050+0000) 2024-05-03T20:53:05.106 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:04 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:04.809+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.627053+0000 front 2024-05-03T20:52:33.627200+0000 (oldest deadline 2024-05-03T20:52:57.127246+0000) 2024-05-03T20:53:05.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:04 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:04.924+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:05.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:04 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:04.924+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:05.171 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:04 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:04.918+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:05.613 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:05 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:53:05.351+0000 7f48bb9b5700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.159238+0000 front 2024-05-03T20:52:27.159389+0000 (oldest deadline 2024-05-03T20:52:50.659055+0000) 2024-05-03T20:53:05.613 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:05 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:53:05.351+0000 7f48bb9b5700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:35.960174+0000 front 2024-05-03T20:52:35.960342+0000 (oldest deadline 2024-05-03T20:53:01.260244+0000) 2024-05-03T20:53:05.614 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:05 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:53:05.351+0000 7f48bb9b5700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.461651+0000 front 2024-05-03T20:52:43.461775+0000 (oldest deadline 2024-05-03T20:53:05.161672+0000) 2024-05-03T20:53:05.614 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:05 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:05.172+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:05.614 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:05 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:05.172+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:05.614 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:05 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:05.172+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:05.712 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:05 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:05.400+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:05.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:05 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:05.400+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:05.713 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:05 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:05.233+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.734848+0000 front 2024-05-03T20:52:27.734877+0000 (oldest deadline 2024-05-03T20:52:50.035068+0000) 2024-05-03T20:53:05.713 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:05 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:05.233+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:35.836984+0000 front 2024-05-03T20:52:35.837125+0000 (oldest deadline 2024-05-03T20:52:59.337050+0000) 2024-05-03T20:53:06.106 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:05 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:05.844+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.627053+0000 front 2024-05-03T20:52:33.627200+0000 (oldest deadline 2024-05-03T20:52:57.127246+0000) 2024-05-03T20:53:06.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:05 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:05.900+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:06.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:05 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:05.900+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:06.314 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:05 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:05.968+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:06.315 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:06 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:06.202+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:06.315 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:06 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:06.202+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:06.315 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:06 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:06.202+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:06.613 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:06 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:53:06.315+0000 7f48bb9b5700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.159238+0000 front 2024-05-03T20:52:27.159389+0000 (oldest deadline 2024-05-03T20:52:50.659055+0000) 2024-05-03T20:53:06.613 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:06 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:53:06.315+0000 7f48bb9b5700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:35.960174+0000 front 2024-05-03T20:52:35.960342+0000 (oldest deadline 2024-05-03T20:53:01.260244+0000) 2024-05-03T20:53:06.613 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:06 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:53:06.315+0000 7f48bb9b5700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.461651+0000 front 2024-05-03T20:52:43.461775+0000 (oldest deadline 2024-05-03T20:53:05.161672+0000) 2024-05-03T20:53:06.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:06 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:06.383+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:06.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:06 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:06.383+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:06.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:06 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:06.383+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:06.713 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:06 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:06.228+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.734848+0000 front 2024-05-03T20:52:27.734877+0000 (oldest deadline 2024-05-03T20:52:50.035068+0000) 2024-05-03T20:53:06.713 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:06 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:06.228+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:35.836984+0000 front 2024-05-03T20:52:35.837125+0000 (oldest deadline 2024-05-03T20:52:59.337050+0000) 2024-05-03T20:53:06.713 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:06 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:06.228+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:39.337636+0000 front 2024-05-03T20:52:39.337822+0000 (oldest deadline 2024-05-03T20:53:05.237655+0000) 2024-05-03T20:53:07.356 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:06 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:06.868+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.627053+0000 front 2024-05-03T20:52:33.627200+0000 (oldest deadline 2024-05-03T20:52:57.127246+0000) 2024-05-03T20:53:07.356 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:06 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:06.894+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:07.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:06 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:06.894+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:07.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:06 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:06.978+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:07.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:06 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:06.978+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:41.415423+0000 front 2024-05-03T20:52:41.415279+0000 (oldest deadline 2024-05-03T20:53:06.115473+0000) 2024-05-03T20:53:07.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:07 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:07.156+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:07.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:07 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:07.156+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:07.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:07 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:07.156+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:07.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:07 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:07.338+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:07.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:07 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:07.338+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:07.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:07 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:07.338+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:07.463 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:07 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:07.204+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.734848+0000 front 2024-05-03T20:52:27.734877+0000 (oldest deadline 2024-05-03T20:52:50.035068+0000) 2024-05-03T20:53:07.463 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:07 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:07.204+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:35.836984+0000 front 2024-05-03T20:52:35.837125+0000 (oldest deadline 2024-05-03T20:52:59.337050+0000) 2024-05-03T20:53:07.463 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:07 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:07.204+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:39.337636+0000 front 2024-05-03T20:52:39.337822+0000 (oldest deadline 2024-05-03T20:53:05.237655+0000) 2024-05-03T20:53:07.709 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:07 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:53:07.364+0000 7f48bb9b5700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.159238+0000 front 2024-05-03T20:52:27.159389+0000 (oldest deadline 2024-05-03T20:52:50.659055+0000) 2024-05-03T20:53:07.709 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:07 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:53:07.364+0000 7f48bb9b5700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:35.960174+0000 front 2024-05-03T20:52:35.960342+0000 (oldest deadline 2024-05-03T20:53:01.260244+0000) 2024-05-03T20:53:07.709 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:07 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5[45387]: 2024-05-03T20:53:07.364+0000 7f48bb9b5700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.461651+0000 front 2024-05-03T20:52:43.461775+0000 (oldest deadline 2024-05-03T20:53:05.161672+0000) 2024-05-03T20:53:07.981 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:07 smithi204 podman[69537]: 2024-05-03 20:53:07.710799733 +0000 UTC m=+5.449786354 container died 3728539110b80c037d1d5c9a5fbc869f12608937d9bdf86536436ec9a0f0c7f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-03T20:53:08.106 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:07 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:07.835+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.627053+0000 front 2024-05-03T20:52:33.627200+0000 (oldest deadline 2024-05-03T20:52:57.127246+0000) 2024-05-03T20:53:08.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:07 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:07.860+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:08.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:07 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:07.860+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:08.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:07 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:07.860+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:08.297 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:08 smithi204 podman[69537]: 2024-05-03 20:53:08.073080271 +0000 UTC m=+5.812066891 container cleanup 3728539110b80c037d1d5c9a5fbc869f12608937d9bdf86536436ec9a0f0c7f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-03T20:53:08.297 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:08 smithi204 bash[69537]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5 2024-05-03T20:53:08.298 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:07 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:07.998+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:08.298 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:07 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:07.998+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:41.415423+0000 front 2024-05-03T20:52:41.415279+0000 (oldest deadline 2024-05-03T20:53:06.115473+0000) 2024-05-03T20:53:08.298 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:08 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:08.122+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:08.298 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:08 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:08.122+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:08.298 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:08 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:08.122+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:08.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:08 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:08.317+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:08.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:08 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:08.317+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:08.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:08 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:08.317+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:08.463 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:08 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:08.160+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.734848+0000 front 2024-05-03T20:52:27.734877+0000 (oldest deadline 2024-05-03T20:52:50.035068+0000) 2024-05-03T20:53:08.463 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:08 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:08.160+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:35.836984+0000 front 2024-05-03T20:52:35.837125+0000 (oldest deadline 2024-05-03T20:52:59.337050+0000) 2024-05-03T20:53:08.464 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:08 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:08.160+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:39.337636+0000 front 2024-05-03T20:52:39.337822+0000 (oldest deadline 2024-05-03T20:53:05.237655+0000) 2024-05-03T20:53:08.590 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:08 smithi204 podman[69554]: 2024-05-03 20:53:08.299260279 +0000 UTC m=+0.584263797 container remove 3728539110b80c037d1d5c9a5fbc869f12608937d9bdf86536436ec9a0f0c7f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T20:53:08.863 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:08 smithi204 podman[69642]: 2024-05-03 20:53:08.591888007 +0000 UTC m=+0.106839719 container create b4d5672e13b4e3a37ae0ba2f7e8c29deff7ccc8993ae363b78546cbac88e88bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430) 2024-05-03T20:53:08.863 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:08 smithi204 podman[69642]: 2024-05-03 20:53:08.503173315 +0000 UTC m=+0.018125007 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T20:53:08.863 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:08 smithi204 podman[69642]: 2024-05-03 20:53:08.803658108 +0000 UTC m=+0.318609788 container init b4d5672e13b4e3a37ae0ba2f7e8c29deff7ccc8993ae363b78546cbac88e88bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True) 2024-05-03T20:53:08.864 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:08 smithi204 podman[69642]: 2024-05-03 20:53:08.807048925 +0000 UTC m=+0.322000599 container start b4d5672e13b4e3a37ae0ba2f7e8c29deff7ccc8993ae363b78546cbac88e88bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True) 2024-05-03T20:53:09.106 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:08 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:08.855+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.627053+0000 front 2024-05-03T20:52:33.627200+0000 (oldest deadline 2024-05-03T20:52:57.127246+0000) 2024-05-03T20:53:09.106 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:08 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:08.855+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:38.228697+0000 front 2024-05-03T20:52:38.228518+0000 (oldest deadline 2024-05-03T20:53:08.228966+0000) 2024-05-03T20:53:09.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:08 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:08.811+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:09.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:08 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:08.811+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:09.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:08 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:08.811+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:09.364 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:08 smithi204 podman[69642]: 2024-05-03 20:53:08.897368044 +0000 UTC m=+0.412319718 container attach b4d5672e13b4e3a37ae0ba2f7e8c29deff7ccc8993ae363b78546cbac88e88bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-03T20:53:09.364 INFO:journalctl@ceph.osd.5.smithi204.stdout:May 03 20:53:09 smithi204 podman[69642]: 2024-05-03 20:53:09.102763122 +0000 UTC m=+0.617714799 container died b4d5672e13b4e3a37ae0ba2f7e8c29deff7ccc8993ae363b78546cbac88e88bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T20:53:09.364 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:08 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:08.979+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:09.364 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:08 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:08.979+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:41.415423+0000 front 2024-05-03T20:52:41.415279+0000 (oldest deadline 2024-05-03T20:53:06.115473+0000) 2024-05-03T20:53:09.365 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:09 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:09.073+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:09.365 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:09 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:09.073+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:09.365 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:09 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:09.073+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:09.462 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:09 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:09.186+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.734848+0000 front 2024-05-03T20:52:27.734877+0000 (oldest deadline 2024-05-03T20:52:50.035068+0000) 2024-05-03T20:53:09.463 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:09 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:09.186+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:35.836984+0000 front 2024-05-03T20:52:35.837125+0000 (oldest deadline 2024-05-03T20:52:59.337050+0000) 2024-05-03T20:53:09.463 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:09 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:09.186+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:39.337636+0000 front 2024-05-03T20:52:39.337822+0000 (oldest deadline 2024-05-03T20:53:05.237655+0000) 2024-05-03T20:53:09.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:09 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:09.333+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:09.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:09 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:09.333+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:09.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:09 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:09.333+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:09.689 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.5.service' 2024-05-03T20:53:09.727 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:53:09.728 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-03T20:53:09.728 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-03T20:53:09.728 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.6 2024-05-03T20:53:10.041 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:09 smithi181 systemd[1]: Stopping Ceph osd.6 for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:53:10.106 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:09 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:09.900+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.627053+0000 front 2024-05-03T20:52:33.627200+0000 (oldest deadline 2024-05-03T20:52:57.127246+0000) 2024-05-03T20:53:10.106 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:09 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:09.900+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:38.228697+0000 front 2024-05-03T20:52:38.228518+0000 (oldest deadline 2024-05-03T20:53:08.228966+0000) 2024-05-03T20:53:10.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:09 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:09.815+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:10.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:09 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:09.815+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:10.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:09 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:09.815+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:10.314 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:10 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:10.042+0000 7fa48b806700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:53:10.314 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:10 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:10.042+0000 7fa48b806700 -1 osd.6 86 *** Got signal Terminated *** 2024-05-03T20:53:10.314 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:10 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:10.042+0000 7fa48b806700 -1 osd.6 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T20:53:10.314 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:10 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:10.194+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.734848+0000 front 2024-05-03T20:52:27.734877+0000 (oldest deadline 2024-05-03T20:52:50.035068+0000) 2024-05-03T20:53:10.315 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:10 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:10.194+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:35.836984+0000 front 2024-05-03T20:52:35.837125+0000 (oldest deadline 2024-05-03T20:52:59.337050+0000) 2024-05-03T20:53:10.315 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:10 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:10.194+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:39.337636+0000 front 2024-05-03T20:52:39.337822+0000 (oldest deadline 2024-05-03T20:53:05.237655+0000) 2024-05-03T20:53:10.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:10 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:10.005+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:10.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:10 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:10.005+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:41.415423+0000 front 2024-05-03T20:52:41.415279+0000 (oldest deadline 2024-05-03T20:53:06.115473+0000) 2024-05-03T20:53:10.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:10 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:10.036+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:10.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:10 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:10.036+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:10.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:10 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:10.036+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:10.712 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:10 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:10.314+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:10.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:10 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:10.314+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:10.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:10 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:10.314+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:11.107 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:10 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:10.900+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.627053+0000 front 2024-05-03T20:52:33.627200+0000 (oldest deadline 2024-05-03T20:52:57.127246+0000) 2024-05-03T20:53:11.107 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:10 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:10.900+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:38.228697+0000 front 2024-05-03T20:52:38.228518+0000 (oldest deadline 2024-05-03T20:53:08.228966+0000) 2024-05-03T20:53:11.108 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:10 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:10.857+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:11.108 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:10 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:10.857+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:11.108 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:10 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:10.857+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:11.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:11 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:11.010+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:11.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:11 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:11.010+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:41.415423+0000 front 2024-05-03T20:52:41.415279+0000 (oldest deadline 2024-05-03T20:53:06.115473+0000) 2024-05-03T20:53:11.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:11 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:11.079+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:11.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:11 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:11.079+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:11.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:11 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:11.079+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:11.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:11 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:11.265+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:11.464 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:11 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:11.265+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:11.464 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:11 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:11.265+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:11.464 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:11 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:11.183+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.734848+0000 front 2024-05-03T20:52:27.734877+0000 (oldest deadline 2024-05-03T20:52:50.035068+0000) 2024-05-03T20:53:11.464 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:11 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:11.183+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:35.836984+0000 front 2024-05-03T20:52:35.837125+0000 (oldest deadline 2024-05-03T20:52:59.337050+0000) 2024-05-03T20:53:11.465 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:11 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:11.183+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:39.337636+0000 front 2024-05-03T20:52:39.337822+0000 (oldest deadline 2024-05-03T20:53:05.237655+0000) 2024-05-03T20:53:12.107 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:11 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:11.888+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.627053+0000 front 2024-05-03T20:52:33.627200+0000 (oldest deadline 2024-05-03T20:52:57.127246+0000) 2024-05-03T20:53:12.108 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:11 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:11.888+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:38.228697+0000 front 2024-05-03T20:52:38.228518+0000 (oldest deadline 2024-05-03T20:53:08.228966+0000) 2024-05-03T20:53:12.109 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:11 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:11.850+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:12.110 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:11 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:11.850+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:12.110 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:11 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:11.850+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:12.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:12 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:12.009+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:12.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:12 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:12.009+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:41.415423+0000 front 2024-05-03T20:52:41.415279+0000 (oldest deadline 2024-05-03T20:53:06.115473+0000) 2024-05-03T20:53:12.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:12 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:12.097+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:12.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:12 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:12.097+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:12.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:12 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:12.097+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:12.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:12 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:12.237+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:12.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:12 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:12.237+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:12.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:12 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:12.237+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:12.463 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:12 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:12.135+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.734848+0000 front 2024-05-03T20:52:27.734877+0000 (oldest deadline 2024-05-03T20:52:50.035068+0000) 2024-05-03T20:53:12.463 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:12 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:12.135+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:35.836984+0000 front 2024-05-03T20:52:35.837125+0000 (oldest deadline 2024-05-03T20:52:59.337050+0000) 2024-05-03T20:53:12.464 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:12 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:12.135+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:39.337636+0000 front 2024-05-03T20:52:39.337822+0000 (oldest deadline 2024-05-03T20:53:05.237655+0000) 2024-05-03T20:53:13.106 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:12 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:12.850+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.627053+0000 front 2024-05-03T20:52:33.627200+0000 (oldest deadline 2024-05-03T20:52:57.127246+0000) 2024-05-03T20:53:13.106 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:12 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:12.850+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:38.228697+0000 front 2024-05-03T20:52:38.228518+0000 (oldest deadline 2024-05-03T20:53:08.228966+0000) 2024-05-03T20:53:13.106 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:12 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:12.833+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:13.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:12 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:12.833+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:13.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:12 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:12.833+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:13.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:12 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:12.985+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:13.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:12 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:12.985+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:41.415423+0000 front 2024-05-03T20:52:41.415279+0000 (oldest deadline 2024-05-03T20:53:06.115473+0000) 2024-05-03T20:53:13.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:13 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:13.101+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:13.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:13 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:13.101+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:13.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:13 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:13.101+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:13.462 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:13 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:13.202+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:13.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:13 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:13.202+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:13.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:13 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:13.202+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:13.463 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:13 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:13.159+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.734848+0000 front 2024-05-03T20:52:27.734877+0000 (oldest deadline 2024-05-03T20:52:50.035068+0000) 2024-05-03T20:53:13.463 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:13 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:13.159+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:35.836984+0000 front 2024-05-03T20:52:35.837125+0000 (oldest deadline 2024-05-03T20:52:59.337050+0000) 2024-05-03T20:53:13.463 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:13 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:13.159+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:39.337636+0000 front 2024-05-03T20:52:39.337822+0000 (oldest deadline 2024-05-03T20:53:05.237655+0000) 2024-05-03T20:53:14.356 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:13 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:13.872+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.627053+0000 front 2024-05-03T20:52:33.627200+0000 (oldest deadline 2024-05-03T20:52:57.127246+0000) 2024-05-03T20:53:14.356 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:13 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:13.872+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:38.228697+0000 front 2024-05-03T20:52:38.228518+0000 (oldest deadline 2024-05-03T20:53:08.228966+0000) 2024-05-03T20:53:14.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:13 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:13.878+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:14.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:13 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:13.878+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:14.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:13 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:13.878+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:14.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:13 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:13.878+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:14.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:13 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:13.940+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:14.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:13 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:13.940+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:41.415423+0000 front 2024-05-03T20:52:41.415279+0000 (oldest deadline 2024-05-03T20:53:06.115473+0000) 2024-05-03T20:53:14.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:14 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:14.135+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:14.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:14 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:14.135+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:14.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:14 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:14.135+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:14.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:14 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:14.203+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:14.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:14 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:14.203+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:14.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:14 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:14.203+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:14.464 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:14 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:14.206+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:27.734848+0000 front 2024-05-03T20:52:27.734877+0000 (oldest deadline 2024-05-03T20:52:50.035068+0000) 2024-05-03T20:53:14.464 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:14 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:14.206+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:35.836984+0000 front 2024-05-03T20:52:35.837125+0000 (oldest deadline 2024-05-03T20:52:59.337050+0000) 2024-05-03T20:53:14.464 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:14 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6[58425]: 2024-05-03T20:53:14.206+0000 7fa487dfa700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:39.337636+0000 front 2024-05-03T20:52:39.337822+0000 (oldest deadline 2024-05-03T20:53:05.237655+0000) 2024-05-03T20:53:15.356 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:14 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:14.869+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.627053+0000 front 2024-05-03T20:52:33.627200+0000 (oldest deadline 2024-05-03T20:52:57.127246+0000) 2024-05-03T20:53:15.356 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:14 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:14.869+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:38.228697+0000 front 2024-05-03T20:52:38.228518+0000 (oldest deadline 2024-05-03T20:53:08.228966+0000) 2024-05-03T20:53:15.357 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:14 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:14.869+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:48.729678+0000 front 2024-05-03T20:52:48.729796+0000 (oldest deadline 2024-05-03T20:53:14.029884+0000) 2024-05-03T20:53:15.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:14 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:14.928+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:15.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:14 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:14.928+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:15.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:14 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:14.928+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:15.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:14 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:14.928+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:15.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:15 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:15.091+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:15.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:15 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:15.091+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:15.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:15 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:15.091+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:15.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:14 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:14.935+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:15.364 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:14 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:14.935+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:41.415423+0000 front 2024-05-03T20:52:41.415279+0000 (oldest deadline 2024-05-03T20:53:06.115473+0000) 2024-05-03T20:53:15.364 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:14 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:14.935+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:50.117064+0000 front 2024-05-03T20:52:50.117080+0000 (oldest deadline 2024-05-03T20:53:14.117507+0000) 2024-05-03T20:53:15.463 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:15 smithi181 podman[91872]: 2024-05-03 20:53:15.410850516 +0000 UTC m=+5.488367439 container died 0a7f20400d87aa8a3d8beb654ef23a54049c9fffd0d4110c874afbe8102bb36f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T20:53:15.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:15 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:15.182+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:15.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:15 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:15.182+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:15.464 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:15 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:15.182+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:15.962 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:15 smithi181 podman[91872]: 2024-05-03 20:53:15.801640756 +0000 UTC m=+5.879157684 container cleanup 0a7f20400d87aa8a3d8beb654ef23a54049c9fffd0d4110c874afbe8102bb36f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T20:53:15.963 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:15 smithi181 bash[91872]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6 2024-05-03T20:53:16.356 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:15 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:15.900+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.627053+0000 front 2024-05-03T20:52:33.627200+0000 (oldest deadline 2024-05-03T20:52:57.127246+0000) 2024-05-03T20:53:16.356 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:15 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:15.900+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:38.228697+0000 front 2024-05-03T20:52:38.228518+0000 (oldest deadline 2024-05-03T20:53:08.228966+0000) 2024-05-03T20:53:16.357 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:15 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:15.900+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:48.729678+0000 front 2024-05-03T20:52:48.729796+0000 (oldest deadline 2024-05-03T20:53:14.029884+0000) 2024-05-03T20:53:16.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:15 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:15.920+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:16.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:15 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:15.920+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:16.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:15 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:15.920+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:16.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:15 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:15.920+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:16.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:16 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:16.045+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:16.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:16 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:16.045+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:16.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:16 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:16.045+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:16.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:15 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:15.937+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:16.364 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:15 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:15.937+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:41.415423+0000 front 2024-05-03T20:52:41.415279+0000 (oldest deadline 2024-05-03T20:53:06.115473+0000) 2024-05-03T20:53:16.364 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:15 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:15.937+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:50.117064+0000 front 2024-05-03T20:52:50.117080+0000 (oldest deadline 2024-05-03T20:53:14.117507+0000) 2024-05-03T20:53:16.418 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:16 smithi181 podman[91901]: 2024-05-03 20:53:16.07781104 +0000 UTC m=+0.656491623 container remove 0a7f20400d87aa8a3d8beb654ef23a54049c9fffd0d4110c874afbe8102bb36f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-03T20:53:16.419 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:16 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:16.161+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:16.419 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:16 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:16.161+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:16.419 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:16 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:16.161+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:16.713 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:16 smithi181 podman[91995]: 2024-05-03 20:53:16.420519288 +0000 UTC m=+0.110011719 container create 325d47736020c993f48ce97ad1f22ccedc649693793bceb22e435e0d8ad63869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T20:53:16.713 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:16 smithi181 podman[91995]: 2024-05-03 20:53:16.328381359 +0000 UTC m=+0.017873811 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T20:53:16.713 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:16 smithi181 podman[91995]: 2024-05-03 20:53:16.648026774 +0000 UTC m=+0.337519206 container init 325d47736020c993f48ce97ad1f22ccedc649693793bceb22e435e0d8ad63869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-03T20:53:16.713 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:16 smithi181 podman[91995]: 2024-05-03 20:53:16.651077417 +0000 UTC m=+0.340569851 container start 325d47736020c993f48ce97ad1f22ccedc649693793bceb22e435e0d8ad63869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2) 2024-05-03T20:53:17.106 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:16 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:16.852+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.627053+0000 front 2024-05-03T20:52:33.627200+0000 (oldest deadline 2024-05-03T20:52:57.127246+0000) 2024-05-03T20:53:17.106 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:16 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:16.852+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:38.228697+0000 front 2024-05-03T20:52:38.228518+0000 (oldest deadline 2024-05-03T20:53:08.228966+0000) 2024-05-03T20:53:17.107 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:16 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:16.852+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:48.729678+0000 front 2024-05-03T20:52:48.729796+0000 (oldest deadline 2024-05-03T20:53:14.029884+0000) 2024-05-03T20:53:17.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:16 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:16.969+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:17.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:16 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:16.969+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:17.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:16 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:16.969+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:17.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:16 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:16.969+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:17.148 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:16 smithi181 podman[91995]: 2024-05-03 20:53:16.73128793 +0000 UTC m=+0.420780375 container attach 325d47736020c993f48ce97ad1f22ccedc649693793bceb22e435e0d8ad63869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-03T20:53:17.148 INFO:journalctl@ceph.osd.6.smithi181.stdout:May 03 20:53:16 smithi181 podman[91995]: 2024-05-03 20:53:16.943142748 +0000 UTC m=+0.632635181 container died 325d47736020c993f48ce97ad1f22ccedc649693793bceb22e435e0d8ad63869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T20:53:17.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:17 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:17.010+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:17.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:17 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:17.010+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:17.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:17 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:17.010+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:17.364 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:16 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:16.910+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:17.364 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:16 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:16.910+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:41.415423+0000 front 2024-05-03T20:52:41.415279+0000 (oldest deadline 2024-05-03T20:53:06.115473+0000) 2024-05-03T20:53:17.364 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:16 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:16.910+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:50.117064+0000 front 2024-05-03T20:52:50.117080+0000 (oldest deadline 2024-05-03T20:53:14.117507+0000) 2024-05-03T20:53:17.462 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:17 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:17.148+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:17.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:17 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:17.148+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:17.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:17 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:17.148+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:17.592 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.6.service' 2024-05-03T20:53:17.630 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:53:17.630 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-03T20:53:17.631 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-03T20:53:17.631 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.7 2024-05-03T20:53:17.926 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:17 smithi196 systemd[1]: Stopping Ceph osd.7 for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:53:17.926 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:17 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:17.805+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.627053+0000 front 2024-05-03T20:52:33.627200+0000 (oldest deadline 2024-05-03T20:52:57.127246+0000) 2024-05-03T20:53:17.926 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:17 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:17.805+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:38.228697+0000 front 2024-05-03T20:52:38.228518+0000 (oldest deadline 2024-05-03T20:53:08.228966+0000) 2024-05-03T20:53:17.927 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:17 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:17.805+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:48.729678+0000 front 2024-05-03T20:52:48.729796+0000 (oldest deadline 2024-05-03T20:53:14.029884+0000) 2024-05-03T20:53:17.927 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:17 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:17.927+0000 7f14020bd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:53:17.927 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:17 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:17.927+0000 7f14020bd700 -1 osd.7 86 *** Got signal Terminated *** 2024-05-03T20:53:17.927 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:17 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:17.927+0000 7f14020bd700 -1 osd.7 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T20:53:18.264 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:17 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:17.950+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:18.264 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:17 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:17.950+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:41.415423+0000 front 2024-05-03T20:52:41.415279+0000 (oldest deadline 2024-05-03T20:53:06.115473+0000) 2024-05-03T20:53:18.264 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:17 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:17.950+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:50.117064+0000 front 2024-05-03T20:52:50.117080+0000 (oldest deadline 2024-05-03T20:53:14.117507+0000) 2024-05-03T20:53:18.265 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:18 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:18.000+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:18.265 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:18 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:18.000+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:18.265 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:18 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:18.000+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:18.356 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:17 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:17.939+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:18.356 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:17 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:17.939+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:18.356 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:17 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:17.939+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:18.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:17 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:17.939+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:18.462 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:18 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:18.178+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:18.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:18 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:18.178+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:18.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:18 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:18.178+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:18.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:18 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:18.178+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:51.660277+0000 front 2024-05-03T20:52:51.660262+0000 (oldest deadline 2024-05-03T20:53:18.061124+0000) 2024-05-03T20:53:19.106 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:18 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:18.803+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.627053+0000 front 2024-05-03T20:52:33.627200+0000 (oldest deadline 2024-05-03T20:52:57.127246+0000) 2024-05-03T20:53:19.107 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:18 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:18.803+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:38.228697+0000 front 2024-05-03T20:52:38.228518+0000 (oldest deadline 2024-05-03T20:53:08.228966+0000) 2024-05-03T20:53:19.107 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:18 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:18.803+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:48.729678+0000 front 2024-05-03T20:52:48.729796+0000 (oldest deadline 2024-05-03T20:53:14.029884+0000) 2024-05-03T20:53:19.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:18 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:18.961+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:19.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:18 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:18.961+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:19.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:18 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:18.961+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:19.108 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:18 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:18.961+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:19.276 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:18 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:18.990+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:19.276 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:18 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:18.990+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:19.276 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:18 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:18.990+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:19.277 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:18 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:18.932+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:19.277 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:18 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:18.932+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:41.415423+0000 front 2024-05-03T20:52:41.415279+0000 (oldest deadline 2024-05-03T20:53:06.115473+0000) 2024-05-03T20:53:19.277 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:18 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:18.932+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:50.117064+0000 front 2024-05-03T20:52:50.117080+0000 (oldest deadline 2024-05-03T20:53:14.117507+0000) 2024-05-03T20:53:19.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:19 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:19.219+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:19.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:19 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:19.219+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:19.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:19 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:19.219+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:19.714 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:19 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:19.219+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:51.660277+0000 front 2024-05-03T20:52:51.660262+0000 (oldest deadline 2024-05-03T20:53:18.061124+0000) 2024-05-03T20:53:20.106 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:19 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:19.807+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.627053+0000 front 2024-05-03T20:52:33.627200+0000 (oldest deadline 2024-05-03T20:52:57.127246+0000) 2024-05-03T20:53:20.106 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:19 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:19.807+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:38.228697+0000 front 2024-05-03T20:52:38.228518+0000 (oldest deadline 2024-05-03T20:53:08.228966+0000) 2024-05-03T20:53:20.107 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:19 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:19.807+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:48.729678+0000 front 2024-05-03T20:52:48.729796+0000 (oldest deadline 2024-05-03T20:53:14.029884+0000) 2024-05-03T20:53:20.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:19 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:19.950+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:20.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:19 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:19.950+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:20.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:19 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:19.950+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:20.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:19 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:19.950+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:20.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:19 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:19.928+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:20.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:19 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:19.928+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:41.415423+0000 front 2024-05-03T20:52:41.415279+0000 (oldest deadline 2024-05-03T20:53:06.115473+0000) 2024-05-03T20:53:20.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:19 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:19.928+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:50.117064+0000 front 2024-05-03T20:52:50.117080+0000 (oldest deadline 2024-05-03T20:53:14.117507+0000) 2024-05-03T20:53:20.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:20 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:20.005+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:20.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:20 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:20.005+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:20.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:20 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:20.005+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:20.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:20 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:20.263+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:20.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:20 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:20.263+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:20.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:20 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:20.263+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:20.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:20 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:20.263+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:51.660277+0000 front 2024-05-03T20:52:51.660262+0000 (oldest deadline 2024-05-03T20:53:18.061124+0000) 2024-05-03T20:53:21.106 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:20 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:20.852+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.627053+0000 front 2024-05-03T20:52:33.627200+0000 (oldest deadline 2024-05-03T20:52:57.127246+0000) 2024-05-03T20:53:21.107 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:20 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:20.852+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:38.228697+0000 front 2024-05-03T20:52:38.228518+0000 (oldest deadline 2024-05-03T20:53:08.228966+0000) 2024-05-03T20:53:21.107 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:20 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:20.852+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:48.729678+0000 front 2024-05-03T20:52:48.729796+0000 (oldest deadline 2024-05-03T20:53:14.029884+0000) 2024-05-03T20:53:21.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:20 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:20.929+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:21.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:20 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:20.929+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:21.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:20 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:20.929+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:21.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:20 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:20.929+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:21.364 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:20 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:20.946+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:21.364 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:20 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:20.946+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:41.415423+0000 front 2024-05-03T20:52:41.415279+0000 (oldest deadline 2024-05-03T20:53:06.115473+0000) 2024-05-03T20:53:21.364 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:20 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:20.946+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:50.117064+0000 front 2024-05-03T20:52:50.117080+0000 (oldest deadline 2024-05-03T20:53:14.117507+0000) 2024-05-03T20:53:21.365 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:20 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:20.996+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:21.365 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:20 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:20.996+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:21.365 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:20 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:20.996+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:21.712 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:21 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:21.241+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:21.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:21 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:21.241+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:21.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:21 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:21.241+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:21.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:21 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:21.241+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:51.660277+0000 front 2024-05-03T20:52:51.660262+0000 (oldest deadline 2024-05-03T20:53:18.061124+0000) 2024-05-03T20:53:22.286 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:21 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:21.914+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:22.287 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:21 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:21.914+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:22.287 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:21 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:21.914+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:22.287 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:21 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:21.914+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:22.287 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:21 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:21.872+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.627053+0000 front 2024-05-03T20:52:33.627200+0000 (oldest deadline 2024-05-03T20:52:57.127246+0000) 2024-05-03T20:53:22.287 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:21 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:21.872+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:38.228697+0000 front 2024-05-03T20:52:38.228518+0000 (oldest deadline 2024-05-03T20:53:08.228966+0000) 2024-05-03T20:53:22.287 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:21 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:21.872+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:48.729678+0000 front 2024-05-03T20:52:48.729796+0000 (oldest deadline 2024-05-03T20:53:14.029884+0000) 2024-05-03T20:53:22.288 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:21 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:21.872+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:57.530853+0000 front 2024-05-03T20:52:57.530885+0000 (oldest deadline 2024-05-03T20:53:21.631011+0000) 2024-05-03T20:53:22.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:22 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:22.002+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:22.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:22 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:22.002+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:22.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:22 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:22.002+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:22.364 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:21 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:21.972+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:22.364 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:21 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:21.972+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:41.415423+0000 front 2024-05-03T20:52:41.415279+0000 (oldest deadline 2024-05-03T20:53:06.115473+0000) 2024-05-03T20:53:22.364 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:21 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:21.972+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:50.117064+0000 front 2024-05-03T20:52:50.117080+0000 (oldest deadline 2024-05-03T20:53:14.117507+0000) 2024-05-03T20:53:22.364 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:21 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:21.972+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.319367+0000 front 2024-05-03T20:52:59.319301+0000 (oldest deadline 2024-05-03T20:53:21.619131+0000) 2024-05-03T20:53:22.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:22 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:22.201+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:22.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:22 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:22.201+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:22.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:22 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:22.201+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:22.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:22 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:22.201+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:51.660277+0000 front 2024-05-03T20:52:51.660262+0000 (oldest deadline 2024-05-03T20:53:18.061124+0000) 2024-05-03T20:53:22.463 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:22 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:22.201+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.562065+0000 front 2024-05-03T20:52:58.562114+0000 (oldest deadline 2024-05-03T20:53:22.061781+0000) 2024-05-03T20:53:23.210 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:22 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:22.893+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:23.211 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:22 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:22.893+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:23.211 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:22 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:22.893+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:23.211 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:22 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:22.893+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:23.211 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:22 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:22.910+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.627053+0000 front 2024-05-03T20:52:33.627200+0000 (oldest deadline 2024-05-03T20:52:57.127246+0000) 2024-05-03T20:53:23.211 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:22 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:22.910+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:38.228697+0000 front 2024-05-03T20:52:38.228518+0000 (oldest deadline 2024-05-03T20:53:08.228966+0000) 2024-05-03T20:53:23.212 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:22 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:22.910+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:48.729678+0000 front 2024-05-03T20:52:48.729796+0000 (oldest deadline 2024-05-03T20:53:14.029884+0000) 2024-05-03T20:53:23.212 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:22 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7[51007]: 2024-05-03T20:53:22.910+0000 7f13fe6b1700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:57.530853+0000 front 2024-05-03T20:52:57.530885+0000 (oldest deadline 2024-05-03T20:53:21.631011+0000) 2024-05-03T20:53:23.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:22 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:22.975+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:23.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:22 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:22.975+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:23.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:22 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:22.975+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:23.364 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:22 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:22.983+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:23.364 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:22 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:22.983+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:41.415423+0000 front 2024-05-03T20:52:41.415279+0000 (oldest deadline 2024-05-03T20:53:06.115473+0000) 2024-05-03T20:53:23.364 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:22 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:22.983+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:50.117064+0000 front 2024-05-03T20:52:50.117080+0000 (oldest deadline 2024-05-03T20:53:14.117507+0000) 2024-05-03T20:53:23.364 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:22 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:22.983+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.319367+0000 front 2024-05-03T20:52:59.319301+0000 (oldest deadline 2024-05-03T20:53:21.619131+0000) 2024-05-03T20:53:23.606 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:23 smithi196 podman[68752]: 2024-05-03 20:53:23.40358024 +0000 UTC m=+5.579830448 container died 94c47cd86dc734c7e58c5a8501f163dbb37e6678815c36dd2ab9741a78bfd841 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430) 2024-05-03T20:53:23.712 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:23 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:23.245+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:23.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:23 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:23.245+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:23.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:23 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:23.245+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:23.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:23 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:23.245+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:51.660277+0000 front 2024-05-03T20:52:51.660262+0000 (oldest deadline 2024-05-03T20:53:18.061124+0000) 2024-05-03T20:53:23.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:23 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:23.245+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.562065+0000 front 2024-05-03T20:52:58.562114+0000 (oldest deadline 2024-05-03T20:53:22.061781+0000) 2024-05-03T20:53:23.926 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:23 smithi196 podman[68752]: 2024-05-03 20:53:23.726784744 +0000 UTC m=+5.903034956 container cleanup 94c47cd86dc734c7e58c5a8501f163dbb37e6678815c36dd2ab9741a78bfd841 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS) 2024-05-03T20:53:23.926 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:23 smithi196 bash[68752]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7 2024-05-03T20:53:23.926 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:23 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:23.868+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:23.926 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:23 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:23.868+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:23.926 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:23 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:23.868+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:23.927 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:23 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:23.868+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:23.927 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:23 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:23.868+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.339124+0000 front 2024-05-03T20:52:58.339158+0000 (oldest deadline 2024-05-03T20:53:23.039197+0000) 2024-05-03T20:53:24.340 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:23 smithi196 podman[68781]: 2024-05-03 20:53:23.927674279 +0000 UTC m=+0.518205069 container remove 94c47cd86dc734c7e58c5a8501f163dbb37e6678815c36dd2ab9741a78bfd841 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-03T20:53:24.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:23 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:23.940+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:24.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:23 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:23.940+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:24.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:23 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:23.940+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:24.364 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:23 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:23.988+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:24.364 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:23 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:23.988+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:41.415423+0000 front 2024-05-03T20:52:41.415279+0000 (oldest deadline 2024-05-03T20:53:06.115473+0000) 2024-05-03T20:53:24.364 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:23 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:23.988+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:50.117064+0000 front 2024-05-03T20:52:50.117080+0000 (oldest deadline 2024-05-03T20:53:14.117507+0000) 2024-05-03T20:53:24.364 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:23 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:23.988+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.319367+0000 front 2024-05-03T20:52:59.319301+0000 (oldest deadline 2024-05-03T20:53:21.619131+0000) 2024-05-03T20:53:24.610 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:24 smithi196 podman[68875]: 2024-05-03 20:53:24.245315379 +0000 UTC m=+0.017871647 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T20:53:24.610 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:24 smithi196 podman[68875]: 2024-05-03 20:53:24.353754192 +0000 UTC m=+0.126310447 container create 87143e022551070f6075d21b3a983f30e8aac051f9f61803cda1ce784b05ceec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-03T20:53:24.610 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:24 smithi196 podman[68875]: 2024-05-03 20:53:24.572638455 +0000 UTC m=+0.345194711 container init 87143e022551070f6075d21b3a983f30e8aac051f9f61803cda1ce784b05ceec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, ceph=True, GIT_CLEAN=True) 2024-05-03T20:53:24.610 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:24 smithi196 podman[68875]: 2024-05-03 20:53:24.576321984 +0000 UTC m=+0.348878247 container start 87143e022551070f6075d21b3a983f30e8aac051f9f61803cda1ce784b05ceec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, ceph=True) 2024-05-03T20:53:24.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:24 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:24.289+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:24.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:24 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:24.289+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:24.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:24 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:24.289+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:24.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:24 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:24.290+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:51.660277+0000 front 2024-05-03T20:52:51.660262+0000 (oldest deadline 2024-05-03T20:53:18.061124+0000) 2024-05-03T20:53:24.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:24 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:24.290+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.562065+0000 front 2024-05-03T20:52:58.562114+0000 (oldest deadline 2024-05-03T20:53:22.061781+0000) 2024-05-03T20:53:24.860 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:24 smithi196 podman[68875]: 2024-05-03 20:53:24.634485231 +0000 UTC m=+0.407041515 container attach 87143e022551070f6075d21b3a983f30e8aac051f9f61803cda1ce784b05ceec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T20:53:25.332 INFO:journalctl@ceph.osd.7.smithi196.stdout:May 03 20:53:24 smithi196 podman[68875]: 2024-05-03 20:53:24.862716779 +0000 UTC m=+0.635273039 container died 87143e022551070f6075d21b3a983f30e8aac051f9f61803cda1ce784b05ceec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430) 2024-05-03T20:53:25.333 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:24 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:24.867+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:25.333 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:24 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:24.867+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:25.333 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:24 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:24.867+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:25.333 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:24 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:24.867+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:25.333 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:24 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:24.867+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.339124+0000 front 2024-05-03T20:52:58.339158+0000 (oldest deadline 2024-05-03T20:53:23.039197+0000) 2024-05-03T20:53:25.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:25 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:25.027+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:25.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:25 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:25.027+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:41.415423+0000 front 2024-05-03T20:52:41.415279+0000 (oldest deadline 2024-05-03T20:53:06.115473+0000) 2024-05-03T20:53:25.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:25 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:25.027+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:50.117064+0000 front 2024-05-03T20:52:50.117080+0000 (oldest deadline 2024-05-03T20:53:14.117507+0000) 2024-05-03T20:53:25.364 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:25 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:25.027+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.319367+0000 front 2024-05-03T20:52:59.319301+0000 (oldest deadline 2024-05-03T20:53:21.619131+0000) 2024-05-03T20:53:25.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:24 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:24.989+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:25.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:24 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:24.989+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:25.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:24 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:24.989+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:25.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:24 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:24.989+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:25.365 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.7.service' 2024-05-03T20:53:25.402 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:53:25.402 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-03T20:53:25.403 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-03T20:53:25.403 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.8 2024-05-03T20:53:25.712 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:25 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:25.325+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:25.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:25 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:25.325+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:25.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:25 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:25.325+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:25.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:25 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:25.325+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:51.660277+0000 front 2024-05-03T20:52:51.660262+0000 (oldest deadline 2024-05-03T20:53:18.061124+0000) 2024-05-03T20:53:25.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:25 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:25.325+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.562065+0000 front 2024-05-03T20:52:58.562114+0000 (oldest deadline 2024-05-03T20:53:22.061781+0000) 2024-05-03T20:53:25.717 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:25 smithi204 systemd[1]: Stopping Ceph osd.8 for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:53:26.025 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:25 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:25.718+0000 7f6c4df83700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:53:26.026 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:25 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:25.718+0000 7f6c4df83700 -1 osd.8 86 *** Got signal Terminated *** 2024-05-03T20:53:26.026 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:25 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:25.718+0000 7f6c4df83700 -1 osd.8 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T20:53:26.356 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:25 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:25.903+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:26.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:25 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:25.903+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:26.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:25 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:25.903+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:26.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:25 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:25.903+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:26.358 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:25 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:25.903+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.339124+0000 front 2024-05-03T20:52:58.339158+0000 (oldest deadline 2024-05-03T20:53:23.039197+0000) 2024-05-03T20:53:26.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:26 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:26.033+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:26.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:26 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:26.033+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:41.415423+0000 front 2024-05-03T20:52:41.415279+0000 (oldest deadline 2024-05-03T20:53:06.115473+0000) 2024-05-03T20:53:26.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:26 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:26.033+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:50.117064+0000 front 2024-05-03T20:52:50.117080+0000 (oldest deadline 2024-05-03T20:53:14.117507+0000) 2024-05-03T20:53:26.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:26 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:26.033+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.319367+0000 front 2024-05-03T20:52:59.319301+0000 (oldest deadline 2024-05-03T20:53:21.619131+0000) 2024-05-03T20:53:26.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:26 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:26.027+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:26.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:26 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:26.027+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:26.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:26 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:26.027+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:26.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:26 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:26.027+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:26.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:26 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:26.277+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:26.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:26 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:26.277+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:26.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:26 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:26.277+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:26.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:26 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:26.277+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:51.660277+0000 front 2024-05-03T20:52:51.660262+0000 (oldest deadline 2024-05-03T20:53:18.061124+0000) 2024-05-03T20:53:26.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:26 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:26.277+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.562065+0000 front 2024-05-03T20:52:58.562114+0000 (oldest deadline 2024-05-03T20:53:22.061781+0000) 2024-05-03T20:53:27.356 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:26 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:26.864+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:27.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:26 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:26.864+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:27.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:26 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:26.864+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:27.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:26 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:26.864+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:27.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:26 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:26.864+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.339124+0000 front 2024-05-03T20:52:58.339158+0000 (oldest deadline 2024-05-03T20:53:23.039197+0000) 2024-05-03T20:53:27.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:27 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:27.053+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:27.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:27 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:27.053+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:41.415423+0000 front 2024-05-03T20:52:41.415279+0000 (oldest deadline 2024-05-03T20:53:06.115473+0000) 2024-05-03T20:53:27.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:27 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:27.053+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:50.117064+0000 front 2024-05-03T20:52:50.117080+0000 (oldest deadline 2024-05-03T20:53:14.117507+0000) 2024-05-03T20:53:27.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:27 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:27.053+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.319367+0000 front 2024-05-03T20:52:59.319301+0000 (oldest deadline 2024-05-03T20:53:21.619131+0000) 2024-05-03T20:53:27.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:26 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:26.986+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:27.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:26 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:26.986+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:27.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:26 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:26.986+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:27.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:26 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:26.986+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:27.644 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:27 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:27.285+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:27.644 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:27 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:27.285+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:27.644 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:27 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:27.285+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:27.645 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:27 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:27.285+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:51.660277+0000 front 2024-05-03T20:52:51.660262+0000 (oldest deadline 2024-05-03T20:53:18.061124+0000) 2024-05-03T20:53:27.645 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:27 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:27.285+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.562065+0000 front 2024-05-03T20:52:58.562114+0000 (oldest deadline 2024-05-03T20:53:22.061781+0000) 2024-05-03T20:53:28.356 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:27 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:27.914+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:28.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:27 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:27.914+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:28.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:27 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:27.914+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:28.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:27 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:27.914+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:28.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:27 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:27.914+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.339124+0000 front 2024-05-03T20:52:58.339158+0000 (oldest deadline 2024-05-03T20:53:23.039197+0000) 2024-05-03T20:53:28.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:28 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:28.049+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:28.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:28 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:28.049+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:41.415423+0000 front 2024-05-03T20:52:41.415279+0000 (oldest deadline 2024-05-03T20:53:06.115473+0000) 2024-05-03T20:53:28.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:28 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:28.049+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:50.117064+0000 front 2024-05-03T20:52:50.117080+0000 (oldest deadline 2024-05-03T20:53:14.117507+0000) 2024-05-03T20:53:28.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:28 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:28.049+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.319367+0000 front 2024-05-03T20:52:59.319301+0000 (oldest deadline 2024-05-03T20:53:21.619131+0000) 2024-05-03T20:53:28.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:27 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:27.979+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:28.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:27 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:27.979+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:28.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:27 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:27.979+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:28.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:27 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:27.979+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:28.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:28 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:28.257+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:28.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:28 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:28.257+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:28.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:28 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:28.257+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:28.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:28 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:28.257+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:51.660277+0000 front 2024-05-03T20:52:51.660262+0000 (oldest deadline 2024-05-03T20:53:18.061124+0000) 2024-05-03T20:53:28.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:28 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:28.257+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.562065+0000 front 2024-05-03T20:52:58.562114+0000 (oldest deadline 2024-05-03T20:53:22.061781+0000) 2024-05-03T20:53:29.356 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:28 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:28.912+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:29.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:28 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:28.912+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:29.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:28 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:28.912+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:29.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:28 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:28.912+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:29.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:28 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:28.912+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.339124+0000 front 2024-05-03T20:52:58.339158+0000 (oldest deadline 2024-05-03T20:53:23.039197+0000) 2024-05-03T20:53:29.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:29 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:29.027+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:29.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:29 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:29.027+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:41.415423+0000 front 2024-05-03T20:52:41.415279+0000 (oldest deadline 2024-05-03T20:53:06.115473+0000) 2024-05-03T20:53:29.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:29 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:29.027+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:50.117064+0000 front 2024-05-03T20:52:50.117080+0000 (oldest deadline 2024-05-03T20:53:14.117507+0000) 2024-05-03T20:53:29.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:29 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:29.027+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.319367+0000 front 2024-05-03T20:52:59.319301+0000 (oldest deadline 2024-05-03T20:53:21.619131+0000) 2024-05-03T20:53:29.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:28 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:28.982+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:29.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:28 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:28.982+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:29.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:28 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:28.982+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:29.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:28 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:28.982+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:29.712 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:29 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:29.249+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:29.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:29 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:29.249+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:29.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:29 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:29.249+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:29.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:29 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:29.249+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:51.660277+0000 front 2024-05-03T20:52:51.660262+0000 (oldest deadline 2024-05-03T20:53:18.061124+0000) 2024-05-03T20:53:29.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:29 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:29.249+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.562065+0000 front 2024-05-03T20:52:58.562114+0000 (oldest deadline 2024-05-03T20:53:22.061781+0000) 2024-05-03T20:53:29.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:29 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:29.249+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:06.063376+0000 front 2024-05-03T20:53:06.063347+0000 (oldest deadline 2024-05-03T20:53:28.963279+0000) 2024-05-03T20:53:30.356 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:29 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:29.906+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:30.356 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:29 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:29.906+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:30.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:29 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:29.906+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:30.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:29 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:29.906+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:30.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:29 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:29.906+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.339124+0000 front 2024-05-03T20:52:58.339158+0000 (oldest deadline 2024-05-03T20:53:23.039197+0000) 2024-05-03T20:53:30.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:30 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:30.018+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:36.214273+0000 front 2024-05-03T20:52:36.214201+0000 (oldest deadline 2024-05-03T20:52:59.714406+0000) 2024-05-03T20:53:30.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:30 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:30.018+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:41.415423+0000 front 2024-05-03T20:52:41.415279+0000 (oldest deadline 2024-05-03T20:53:06.115473+0000) 2024-05-03T20:53:30.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:30 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:30.018+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:50.117064+0000 front 2024-05-03T20:52:50.117080+0000 (oldest deadline 2024-05-03T20:53:14.117507+0000) 2024-05-03T20:53:30.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:30 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8[51956]: 2024-05-03T20:53:30.018+0000 7f6c4656f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.319367+0000 front 2024-05-03T20:52:59.319301+0000 (oldest deadline 2024-05-03T20:53:21.619131+0000) 2024-05-03T20:53:30.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:29 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:29.956+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:30.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:29 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:29.956+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:30.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:29 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:29.956+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:30.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:29 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:29.956+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:30.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:30 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:30.225+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:30.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:30 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:30.225+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:30.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:30 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:30.225+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:30.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:30 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:30.225+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:51.660277+0000 front 2024-05-03T20:52:51.660262+0000 (oldest deadline 2024-05-03T20:53:18.061124+0000) 2024-05-03T20:53:30.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:30 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:30.225+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.562065+0000 front 2024-05-03T20:52:58.562114+0000 (oldest deadline 2024-05-03T20:53:22.061781+0000) 2024-05-03T20:53:30.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:30 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:30.225+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:06.063376+0000 front 2024-05-03T20:53:06.063347+0000 (oldest deadline 2024-05-03T20:53:28.963279+0000) 2024-05-03T20:53:31.351 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:31 smithi204 podman[69761]: 2024-05-03 20:53:31.084177327 +0000 UTC m=+5.489753639 container died ab3c3d9c2e60096cc5afe38cc7cc8d07f5a0a97fce638f0f2deca962a1aa24f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-05-03T20:53:31.352 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:30 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:30.910+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:31.352 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:30 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:30.910+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:31.352 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:30 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:30.910+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:31.352 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:30 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:30.910+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:31.356 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:30 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:30.897+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:31.356 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:30 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:30.897+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:31.356 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:30 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:30.897+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:31.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:30 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:30.897+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:31.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:30 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:30.897+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.339124+0000 front 2024-05-03T20:52:58.339158+0000 (oldest deadline 2024-05-03T20:53:23.039197+0000) 2024-05-03T20:53:31.613 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:31 smithi204 podman[69761]: 2024-05-03 20:53:31.451635044 +0000 UTC m=+5.857211355 container cleanup ab3c3d9c2e60096cc5afe38cc7cc8d07f5a0a97fce638f0f2deca962a1aa24f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T20:53:31.613 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:31 smithi204 bash[69761]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8 2024-05-03T20:53:31.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:31 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:31.243+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:31.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:31 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:31.243+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:31.714 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:31 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:31.243+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:31.714 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:31 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:31.243+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:51.660277+0000 front 2024-05-03T20:52:51.660262+0000 (oldest deadline 2024-05-03T20:53:18.061124+0000) 2024-05-03T20:53:31.715 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:31 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:31.243+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.562065+0000 front 2024-05-03T20:52:58.562114+0000 (oldest deadline 2024-05-03T20:53:22.061781+0000) 2024-05-03T20:53:31.715 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:31 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:31.243+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:06.063376+0000 front 2024-05-03T20:53:06.063347+0000 (oldest deadline 2024-05-03T20:53:28.963279+0000) 2024-05-03T20:53:32.068 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:31 smithi204 podman[69779]: 2024-05-03 20:53:31.719651198 +0000 UTC m=+0.632485853 container remove ab3c3d9c2e60096cc5afe38cc7cc8d07f5a0a97fce638f0f2deca962a1aa24f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T20:53:32.069 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:31 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:31.890+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:32.069 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:31 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:31.890+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:32.069 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:31 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:31.890+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:32.069 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:31 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:31.890+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:32.356 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:31 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:31.885+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:32.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:31 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:31.885+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:32.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:31 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:31.885+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:32.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:31 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:31.885+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:32.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:31 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:31.885+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.339124+0000 front 2024-05-03T20:52:58.339158+0000 (oldest deadline 2024-05-03T20:53:23.039197+0000) 2024-05-03T20:53:32.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:31 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:31.885+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:05.940276+0000 front 2024-05-03T20:53:05.940241+0000 (oldest deadline 2024-05-03T20:53:31.740485+0000) 2024-05-03T20:53:32.362 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:32 smithi204 podman[69868]: 2024-05-03 20:53:32.070416308 +0000 UTC m=+0.110890018 container create c75028d17aee450c1ce306c477bfbaefbe1fafc944f6744ebeb8f0f249c5106c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=) 2024-05-03T20:53:32.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:32 smithi204 podman[69868]: 2024-05-03 20:53:31.977297943 +0000 UTC m=+0.017771666 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T20:53:32.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:32 smithi204 podman[69868]: 2024-05-03 20:53:32.281305402 +0000 UTC m=+0.321779112 container init c75028d17aee450c1ce306c477bfbaefbe1fafc944f6744ebeb8f0f249c5106c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-05-03T20:53:32.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:32 smithi204 podman[69868]: 2024-05-03 20:53:32.284679202 +0000 UTC m=+0.325152913 container start c75028d17aee450c1ce306c477bfbaefbe1fafc944f6744ebeb8f0f249c5106c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-03T20:53:32.363 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:32 smithi204 podman[69868]: 2024-05-03 20:53:32.364509905 +0000 UTC m=+0.404983618 container attach c75028d17aee450c1ce306c477bfbaefbe1fafc944f6744ebeb8f0f249c5106c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-05-03T20:53:32.613 INFO:journalctl@ceph.osd.8.smithi204.stdout:May 03 20:53:32 smithi204 podman[69868]: 2024-05-03 20:53:32.590566484 +0000 UTC m=+0.631040199 container died c75028d17aee450c1ce306c477bfbaefbe1fafc944f6744ebeb8f0f249c5106c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-03T20:53:32.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:32 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:32.291+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:32.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:32 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:32.291+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:32.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:32 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:32.291+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:32.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:32 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:32.291+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:51.660277+0000 front 2024-05-03T20:52:51.660262+0000 (oldest deadline 2024-05-03T20:53:18.061124+0000) 2024-05-03T20:53:32.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:32 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:32.291+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.562065+0000 front 2024-05-03T20:52:58.562114+0000 (oldest deadline 2024-05-03T20:53:22.061781+0000) 2024-05-03T20:53:32.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:32 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:32.291+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:06.063376+0000 front 2024-05-03T20:53:06.063347+0000 (oldest deadline 2024-05-03T20:53:28.963279+0000) 2024-05-03T20:53:33.158 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:32 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:32.885+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:33.159 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:32 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:32.885+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:33.159 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:32 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:32.885+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:33.159 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:32 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:32.885+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:33.177 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.8.service' 2024-05-03T20:53:33.215 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:53:33.216 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-03T20:53:33.216 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-03T20:53:33.216 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.9 2024-05-03T20:53:33.356 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:32 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:32.930+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:33.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:32 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:32.930+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:33.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:32 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:32.930+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:33.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:32 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:32.930+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:33.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:32 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:32.930+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.339124+0000 front 2024-05-03T20:52:58.339158+0000 (oldest deadline 2024-05-03T20:53:23.039197+0000) 2024-05-03T20:53:33.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:32 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:32.930+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:05.940276+0000 front 2024-05-03T20:53:05.940241+0000 (oldest deadline 2024-05-03T20:53:31.740485+0000) 2024-05-03T20:53:33.512 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:33 smithi181 systemd[1]: Stopping Ceph osd.9 for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:53:33.512 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:33 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:33.309+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:33.512 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:33 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:33.309+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:33.512 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:33 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:33.309+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:33.512 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:33 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:33.309+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:51.660277+0000 front 2024-05-03T20:52:51.660262+0000 (oldest deadline 2024-05-03T20:53:18.061124+0000) 2024-05-03T20:53:33.512 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:33 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:33.309+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.562065+0000 front 2024-05-03T20:52:58.562114+0000 (oldest deadline 2024-05-03T20:53:22.061781+0000) 2024-05-03T20:53:33.513 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:33 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:33.309+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:06.063376+0000 front 2024-05-03T20:53:06.063347+0000 (oldest deadline 2024-05-03T20:53:28.963279+0000) 2024-05-03T20:53:33.513 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:33 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:33.513+0000 7f96663e5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:53:33.513 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:33 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:33.513+0000 7f96663e5700 -1 osd.9 86 *** Got signal Terminated *** 2024-05-03T20:53:33.513 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:33 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:33.513+0000 7f96663e5700 -1 osd.9 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T20:53:34.351 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:33 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:33.904+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:34.351 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:33 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:33.904+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:34.351 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:33 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:33.904+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:34.351 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:33 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:33.904+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:34.351 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:33 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:33.904+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:07.197190+0000 front 2024-05-03T20:53:07.197123+0000 (oldest deadline 2024-05-03T20:53:33.097275+0000) 2024-05-03T20:53:34.356 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:33 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:33.972+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:34.356 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:33 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:33.972+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:34.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:33 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:33.972+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:34.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:33 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:33.972+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:34.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:33 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:33.972+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.339124+0000 front 2024-05-03T20:52:58.339158+0000 (oldest deadline 2024-05-03T20:53:23.039197+0000) 2024-05-03T20:53:34.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:33 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:33.972+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:05.940276+0000 front 2024-05-03T20:53:05.940241+0000 (oldest deadline 2024-05-03T20:53:31.740485+0000) 2024-05-03T20:53:34.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:34 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:34.350+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:34.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:34 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:34.350+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:34.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:34 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:34.350+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:34.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:34 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:34.350+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:51.660277+0000 front 2024-05-03T20:52:51.660262+0000 (oldest deadline 2024-05-03T20:53:18.061124+0000) 2024-05-03T20:53:34.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:34 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:34.350+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.562065+0000 front 2024-05-03T20:52:58.562114+0000 (oldest deadline 2024-05-03T20:53:22.061781+0000) 2024-05-03T20:53:34.714 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:34 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:34.350+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:06.063376+0000 front 2024-05-03T20:53:06.063347+0000 (oldest deadline 2024-05-03T20:53:28.963279+0000) 2024-05-03T20:53:35.356 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:34 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:34.933+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:35.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:34 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:34.933+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:35.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:34 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:34.933+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:35.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:34 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:34.933+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:35.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:34 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:34.933+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.339124+0000 front 2024-05-03T20:52:58.339158+0000 (oldest deadline 2024-05-03T20:53:23.039197+0000) 2024-05-03T20:53:35.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:34 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:34.933+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:05.940276+0000 front 2024-05-03T20:53:05.940241+0000 (oldest deadline 2024-05-03T20:53:31.740485+0000) 2024-05-03T20:53:35.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:34 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:34.876+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:35.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:34 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:34.876+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:35.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:34 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:34.876+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:35.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:34 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:34.876+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:35.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:34 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:34.876+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:07.197190+0000 front 2024-05-03T20:53:07.197123+0000 (oldest deadline 2024-05-03T20:53:33.097275+0000) 2024-05-03T20:53:35.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:35 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:35.359+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:35.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:35 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:35.359+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:35.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:35 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:35.359+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:35.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:35 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:35.359+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:51.660277+0000 front 2024-05-03T20:52:51.660262+0000 (oldest deadline 2024-05-03T20:53:18.061124+0000) 2024-05-03T20:53:35.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:35 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:35.359+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.562065+0000 front 2024-05-03T20:52:58.562114+0000 (oldest deadline 2024-05-03T20:53:22.061781+0000) 2024-05-03T20:53:35.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:35 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:35.359+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:06.063376+0000 front 2024-05-03T20:53:06.063347+0000 (oldest deadline 2024-05-03T20:53:28.963279+0000) 2024-05-03T20:53:36.356 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:35 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:35.894+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:36.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:35 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:35.894+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:36.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:35 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:35.894+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:36.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:35 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:35.894+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:36.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:35 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:35.894+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.339124+0000 front 2024-05-03T20:52:58.339158+0000 (oldest deadline 2024-05-03T20:53:23.039197+0000) 2024-05-03T20:53:36.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:35 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:35.894+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:05.940276+0000 front 2024-05-03T20:53:05.940241+0000 (oldest deadline 2024-05-03T20:53:31.740485+0000) 2024-05-03T20:53:36.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:35 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:35.879+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:36.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:35 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:35.879+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:36.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:35 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:35.879+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:36.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:35 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:35.879+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:36.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:35 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:35.879+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:07.197190+0000 front 2024-05-03T20:53:07.197123+0000 (oldest deadline 2024-05-03T20:53:33.097275+0000) 2024-05-03T20:53:36.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:36 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:36.350+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:36.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:36 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:36.350+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:36.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:36 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:36.350+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:36.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:36 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:36.350+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:51.660277+0000 front 2024-05-03T20:52:51.660262+0000 (oldest deadline 2024-05-03T20:53:18.061124+0000) 2024-05-03T20:53:36.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:36 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:36.350+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.562065+0000 front 2024-05-03T20:52:58.562114+0000 (oldest deadline 2024-05-03T20:53:22.061781+0000) 2024-05-03T20:53:36.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:36 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:36.350+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:06.063376+0000 front 2024-05-03T20:53:06.063347+0000 (oldest deadline 2024-05-03T20:53:28.963279+0000) 2024-05-03T20:53:37.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:36 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:36.836+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:37.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:36 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:36.836+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:37.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:36 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:36.836+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:37.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:36 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:36.836+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:37.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:36 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:36.836+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:07.197190+0000 front 2024-05-03T20:53:07.197123+0000 (oldest deadline 2024-05-03T20:53:33.097275+0000) 2024-05-03T20:53:37.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:36 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:36.836+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6822 osd.6 since back 2024-05-03T20:53:13.097786+0000 front 2024-05-03T20:53:13.097805+0000 (oldest deadline 2024-05-03T20:53:36.597842+0000) 2024-05-03T20:53:37.356 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:36 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:36.884+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:37.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:36 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:36.884+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:37.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:36 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:36.884+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:37.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:36 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:36.884+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:37.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:36 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:36.884+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.339124+0000 front 2024-05-03T20:52:58.339158+0000 (oldest deadline 2024-05-03T20:53:23.039197+0000) 2024-05-03T20:53:37.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:36 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:36.884+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:05.940276+0000 front 2024-05-03T20:53:05.940241+0000 (oldest deadline 2024-05-03T20:53:31.740485+0000) 2024-05-03T20:53:37.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:37 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:37.379+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:37.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:37 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:37.379+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:37.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:37 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:37.379+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:37.714 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:37 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:37.379+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:51.660277+0000 front 2024-05-03T20:52:51.660262+0000 (oldest deadline 2024-05-03T20:53:18.061124+0000) 2024-05-03T20:53:37.714 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:37 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:37.379+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.562065+0000 front 2024-05-03T20:52:58.562114+0000 (oldest deadline 2024-05-03T20:53:22.061781+0000) 2024-05-03T20:53:37.714 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:37 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:37.379+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:06.063376+0000 front 2024-05-03T20:53:06.063347+0000 (oldest deadline 2024-05-03T20:53:28.963279+0000) 2024-05-03T20:53:38.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:37 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:37.790+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:38.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:37 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:37.790+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:38.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:37 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:37.790+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:38.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:37 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:37.790+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:38.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:37 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:37.790+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:07.197190+0000 front 2024-05-03T20:53:07.197123+0000 (oldest deadline 2024-05-03T20:53:33.097275+0000) 2024-05-03T20:53:38.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:37 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:37.790+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6822 osd.6 since back 2024-05-03T20:53:13.097786+0000 front 2024-05-03T20:53:13.097805+0000 (oldest deadline 2024-05-03T20:53:36.597842+0000) 2024-05-03T20:53:38.356 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:37 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:37.871+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:38.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:37 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:37.871+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:38.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:37 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:37.871+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:38.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:37 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:37.871+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:38.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:37 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:37.871+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.339124+0000 front 2024-05-03T20:52:58.339158+0000 (oldest deadline 2024-05-03T20:53:23.039197+0000) 2024-05-03T20:53:38.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:37 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:37.871+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:05.940276+0000 front 2024-05-03T20:53:05.940241+0000 (oldest deadline 2024-05-03T20:53:31.740485+0000) 2024-05-03T20:53:38.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:38 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:38.339+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:28.356701+0000 front 2024-05-03T20:52:28.356802+0000 (oldest deadline 2024-05-03T20:52:53.056945+0000) 2024-05-03T20:53:38.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:38 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:38.339+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:33.558008+0000 front 2024-05-03T20:52:33.557876+0000 (oldest deadline 2024-05-03T20:52:59.457891+0000) 2024-05-03T20:53:38.713 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:38 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:38.339+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:42.358968+0000 front 2024-05-03T20:52:42.358851+0000 (oldest deadline 2024-05-03T20:53:05.858742+0000) 2024-05-03T20:53:38.714 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:38 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:38.339+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:51.660277+0000 front 2024-05-03T20:52:51.660262+0000 (oldest deadline 2024-05-03T20:53:18.061124+0000) 2024-05-03T20:53:38.714 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:38 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:38.339+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.562065+0000 front 2024-05-03T20:52:58.562114+0000 (oldest deadline 2024-05-03T20:53:22.061781+0000) 2024-05-03T20:53:38.714 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:38 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9[65040]: 2024-05-03T20:53:38.339+0000 7f965e9d1700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:06.063376+0000 front 2024-05-03T20:53:06.063347+0000 (oldest deadline 2024-05-03T20:53:28.963279+0000) 2024-05-03T20:53:39.110 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:38 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:38.835+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:39.111 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:38 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:38.835+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:39.111 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:38 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:38.835+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:39.111 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:38 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:38.835+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:39.111 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:38 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:38.835+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.339124+0000 front 2024-05-03T20:52:58.339158+0000 (oldest deadline 2024-05-03T20:53:23.039197+0000) 2024-05-03T20:53:39.111 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:38 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:38.835+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:05.940276+0000 front 2024-05-03T20:53:05.940241+0000 (oldest deadline 2024-05-03T20:53:31.740485+0000) 2024-05-03T20:53:39.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:38 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:38.827+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:39.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:38 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:38.827+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:39.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:38 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:38.827+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:39.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:38 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:38.827+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:39.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:38 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:38.827+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:07.197190+0000 front 2024-05-03T20:53:07.197123+0000 (oldest deadline 2024-05-03T20:53:33.097275+0000) 2024-05-03T20:53:39.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:38 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:38.827+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6822 osd.6 since back 2024-05-03T20:53:13.097786+0000 front 2024-05-03T20:53:13.097805+0000 (oldest deadline 2024-05-03T20:53:36.597842+0000) 2024-05-03T20:53:39.245 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:38 smithi181 podman[92101]: 2024-05-03 20:53:38.980306825 +0000 UTC m=+5.571972753 container died 774db69cb90ae275c43771ea7b738774d12fae15e733ef5b7455cd4abc8dab39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T20:53:39.594 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:39 smithi181 podman[92101]: 2024-05-03 20:53:39.345517513 +0000 UTC m=+5.937183481 container cleanup 774db69cb90ae275c43771ea7b738774d12fae15e733ef5b7455cd4abc8dab39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-03T20:53:39.595 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:39 smithi181 bash[92101]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9 2024-05-03T20:53:39.909 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:39 smithi181 podman[92118]: 2024-05-03 20:53:39.596538421 +0000 UTC m=+0.611732416 container remove 774db69cb90ae275c43771ea7b738774d12fae15e733ef5b7455cd4abc8dab39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T20:53:40.207 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:39 smithi181 podman[92213]: 2024-05-03 20:53:39.814171019 +0000 UTC m=+0.017973355 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T20:53:40.208 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:39 smithi181 podman[92213]: 2024-05-03 20:53:39.931148369 +0000 UTC m=+0.134950680 container create fe6cae4287a2cf91847508539d78f662a348aa87b0ad0a30be8875b7ac6fa9d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9-deactivate, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T20:53:40.208 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:40 smithi181 podman[92213]: 2024-05-03 20:53:40.133335653 +0000 UTC m=+0.337137962 container init fe6cae4287a2cf91847508539d78f662a348aa87b0ad0a30be8875b7ac6fa9d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430) 2024-05-03T20:53:40.208 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:40 smithi181 podman[92213]: 2024-05-03 20:53:40.136429047 +0000 UTC m=+0.340231360 container start fe6cae4287a2cf91847508539d78f662a348aa87b0ad0a30be8875b7ac6fa9d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430) 2024-05-03T20:53:40.208 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:40 smithi181 podman[92213]: 2024-05-03 20:53:40.208371551 +0000 UTC m=+0.412173890 container attach fe6cae4287a2cf91847508539d78f662a348aa87b0ad0a30be8875b7ac6fa9d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T20:53:40.356 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:39 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:39.875+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:40.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:39 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:39.875+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:40.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:39 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:39.875+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:40.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:39 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:39.875+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:40.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:39 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:39.875+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.339124+0000 front 2024-05-03T20:52:58.339158+0000 (oldest deadline 2024-05-03T20:53:23.039197+0000) 2024-05-03T20:53:40.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:39 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:39.875+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:05.940276+0000 front 2024-05-03T20:53:05.940241+0000 (oldest deadline 2024-05-03T20:53:31.740485+0000) 2024-05-03T20:53:40.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:39 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:39.867+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:40.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:39 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:39.867+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:40.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:39 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:39.867+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:40.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:39 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:39.867+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:40.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:39 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:39.867+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:07.197190+0000 front 2024-05-03T20:53:07.197123+0000 (oldest deadline 2024-05-03T20:53:33.097275+0000) 2024-05-03T20:53:40.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:39 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:39.867+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6822 osd.6 since back 2024-05-03T20:53:13.097786+0000 front 2024-05-03T20:53:13.097805+0000 (oldest deadline 2024-05-03T20:53:36.597842+0000) 2024-05-03T20:53:40.462 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:40 smithi181 podman[92213]: 2024-05-03 20:53:40.427100341 +0000 UTC m=+0.630902676 container died fe6cae4287a2cf91847508539d78f662a348aa87b0ad0a30be8875b7ac6fa9d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD) 2024-05-03T20:53:40.950 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.9.service' 2024-05-03T20:53:40.971 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:40 smithi181 podman[92239]: 2024-05-03 20:53:40.936784654 +0000 UTC m=+0.504452007 container remove fe6cae4287a2cf91847508539d78f662a348aa87b0ad0a30be8875b7ac6fa9d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-03T20:53:40.972 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:40 smithi181 systemd[1]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.9.service: Deactivated successfully. 2024-05-03T20:53:40.972 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:40 smithi181 systemd[1]: Stopped Ceph osd.9 for 1cd058ee-098b-11ef-bc97-c7b262605968. 2024-05-03T20:53:40.972 INFO:journalctl@ceph.osd.9.smithi181.stdout:May 03 20:53:40 smithi181 systemd[1]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.9.service: Consumed 10.226s CPU time. 2024-05-03T20:53:40.986 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:53:40.986 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-03T20:53:40.987 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-03T20:53:40.987 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.10 2024-05-03T20:53:41.106 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:40 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:40.848+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:41.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:40 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:40.848+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:41.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:40 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:40.848+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:41.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:40 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:40.848+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:41.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:40 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:40.848+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.339124+0000 front 2024-05-03T20:52:58.339158+0000 (oldest deadline 2024-05-03T20:53:23.039197+0000) 2024-05-03T20:53:41.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:40 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:40.848+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:05.940276+0000 front 2024-05-03T20:53:05.940241+0000 (oldest deadline 2024-05-03T20:53:31.740485+0000) 2024-05-03T20:53:41.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:40 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:40.848+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6822 osd.6 since back 2024-05-03T20:53:11.740981+0000 front 2024-05-03T20:53:11.740962+0000 (oldest deadline 2024-05-03T20:53:39.941596+0000) 2024-05-03T20:53:41.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:41 smithi196 systemd[1]: Stopping Ceph osd.10 for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:53:41.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:40 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:40.848+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:41.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:40 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:40.848+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:41.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:40 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:40.848+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:41.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:40 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:40.848+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:41.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:40 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:40.848+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:07.197190+0000 front 2024-05-03T20:53:07.197123+0000 (oldest deadline 2024-05-03T20:53:33.097275+0000) 2024-05-03T20:53:41.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:40 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:40.848+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6822 osd.6 since back 2024-05-03T20:53:13.097786+0000 front 2024-05-03T20:53:13.097805+0000 (oldest deadline 2024-05-03T20:53:36.597842+0000) 2024-05-03T20:53:41.606 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:41 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:41.302+0000 7f48f5010700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:53:41.607 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:41 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:41.302+0000 7f48f5010700 -1 osd.10 86 *** Got signal Terminated *** 2024-05-03T20:53:41.607 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:41 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:41.302+0000 7f48f5010700 -1 osd.10 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T20:53:42.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:41 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:41.844+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:42.159 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:41 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:41.844+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:42.159 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:41 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:41.844+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:42.159 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:41 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:41.844+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:42.160 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:41 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:41.844+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:07.197190+0000 front 2024-05-03T20:53:07.197123+0000 (oldest deadline 2024-05-03T20:53:33.097275+0000) 2024-05-03T20:53:42.160 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:41 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:41.844+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6822 osd.6 since back 2024-05-03T20:53:13.097786+0000 front 2024-05-03T20:53:13.097805+0000 (oldest deadline 2024-05-03T20:53:36.597842+0000) 2024-05-03T20:53:42.356 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:41 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:41.885+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:42.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:41 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:41.885+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:42.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:41 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:41.885+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:42.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:41 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:41.885+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:42.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:41 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:41.885+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.339124+0000 front 2024-05-03T20:52:58.339158+0000 (oldest deadline 2024-05-03T20:53:23.039197+0000) 2024-05-03T20:53:42.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:41 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:41.885+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:05.940276+0000 front 2024-05-03T20:53:05.940241+0000 (oldest deadline 2024-05-03T20:53:31.740485+0000) 2024-05-03T20:53:42.357 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:41 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:41.885+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6822 osd.6 since back 2024-05-03T20:53:11.740981+0000 front 2024-05-03T20:53:11.740962+0000 (oldest deadline 2024-05-03T20:53:39.941596+0000) 2024-05-03T20:53:43.106 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:42 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:42.839+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:43.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:42 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:42.839+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:43.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:42 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:42.839+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:43.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:42 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:42.839+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:43.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:42 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:42.839+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.339124+0000 front 2024-05-03T20:52:58.339158+0000 (oldest deadline 2024-05-03T20:53:23.039197+0000) 2024-05-03T20:53:43.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:42 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:42.839+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:05.940276+0000 front 2024-05-03T20:53:05.940241+0000 (oldest deadline 2024-05-03T20:53:31.740485+0000) 2024-05-03T20:53:43.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:42 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:42.839+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6822 osd.6 since back 2024-05-03T20:53:11.740981+0000 front 2024-05-03T20:53:11.740962+0000 (oldest deadline 2024-05-03T20:53:39.941596+0000) 2024-05-03T20:53:43.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:42 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:42.817+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:43.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:42 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:42.817+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:43.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:42 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:42.817+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:43.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:42 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:42.817+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:43.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:42 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:42.817+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:07.197190+0000 front 2024-05-03T20:53:07.197123+0000 (oldest deadline 2024-05-03T20:53:33.097275+0000) 2024-05-03T20:53:43.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:42 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:42.817+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6822 osd.6 since back 2024-05-03T20:53:13.097786+0000 front 2024-05-03T20:53:13.097805+0000 (oldest deadline 2024-05-03T20:53:36.597842+0000) 2024-05-03T20:53:44.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:43 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:43.847+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:44.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:43 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:43.847+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:44.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:43 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:43.847+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:44.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:43 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:43.847+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:44.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:43 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:43.847+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.339124+0000 front 2024-05-03T20:52:58.339158+0000 (oldest deadline 2024-05-03T20:53:23.039197+0000) 2024-05-03T20:53:44.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:43 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:43.847+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:05.940276+0000 front 2024-05-03T20:53:05.940241+0000 (oldest deadline 2024-05-03T20:53:31.740485+0000) 2024-05-03T20:53:44.108 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:43 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:43.847+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6822 osd.6 since back 2024-05-03T20:53:11.740981+0000 front 2024-05-03T20:53:11.740962+0000 (oldest deadline 2024-05-03T20:53:39.941596+0000) 2024-05-03T20:53:44.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:43 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:43.789+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:44.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:43 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:43.789+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:44.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:43 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:43.789+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:44.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:43 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:43.789+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:44.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:43 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:43.789+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:07.197190+0000 front 2024-05-03T20:53:07.197123+0000 (oldest deadline 2024-05-03T20:53:33.097275+0000) 2024-05-03T20:53:44.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:43 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:43.789+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6822 osd.6 since back 2024-05-03T20:53:13.097786+0000 front 2024-05-03T20:53:13.097805+0000 (oldest deadline 2024-05-03T20:53:36.597842+0000) 2024-05-03T20:53:45.106 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:44 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:44.827+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:45.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:44 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:44.827+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:45.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:44 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:44.827+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:45.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:44 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:44.827+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:45.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:44 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:44.827+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.339124+0000 front 2024-05-03T20:52:58.339158+0000 (oldest deadline 2024-05-03T20:53:23.039197+0000) 2024-05-03T20:53:45.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:44 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:44.827+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:05.940276+0000 front 2024-05-03T20:53:05.940241+0000 (oldest deadline 2024-05-03T20:53:31.740485+0000) 2024-05-03T20:53:45.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:44 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:44.827+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6822 osd.6 since back 2024-05-03T20:53:11.740981+0000 front 2024-05-03T20:53:11.740962+0000 (oldest deadline 2024-05-03T20:53:39.941596+0000) 2024-05-03T20:53:45.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:44 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:44.740+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:45.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:44 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:44.740+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:45.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:44 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:44.740+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:45.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:44 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:44.740+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:45.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:44 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:44.740+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:07.197190+0000 front 2024-05-03T20:53:07.197123+0000 (oldest deadline 2024-05-03T20:53:33.097275+0000) 2024-05-03T20:53:45.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:44 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:44.740+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6822 osd.6 since back 2024-05-03T20:53:13.097786+0000 front 2024-05-03T20:53:13.097805+0000 (oldest deadline 2024-05-03T20:53:36.597842+0000) 2024-05-03T20:53:46.106 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:45 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:45.850+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:25.133826+0000 front 2024-05-03T20:52:25.134007+0000 (oldest deadline 2024-05-03T20:52:50.434044+0000) 2024-05-03T20:53:46.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:45 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:45.850+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.535208+0000 front 2024-05-03T20:52:34.535252+0000 (oldest deadline 2024-05-03T20:52:56.835368+0000) 2024-05-03T20:53:46.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:45 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:45.850+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:43.236858+0000 front 2024-05-03T20:52:43.236848+0000 (oldest deadline 2024-05-03T20:53:07.336953+0000) 2024-05-03T20:53:46.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:45 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:45.850+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6814 osd.3 since back 2024-05-03T20:52:47.837978+0000 front 2024-05-03T20:52:47.837861+0000 (oldest deadline 2024-05-03T20:53:13.137953+0000) 2024-05-03T20:53:46.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:45 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:45.850+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:58.339124+0000 front 2024-05-03T20:52:58.339158+0000 (oldest deadline 2024-05-03T20:53:23.039197+0000) 2024-05-03T20:53:46.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:45 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:45.850+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:05.940276+0000 front 2024-05-03T20:53:05.940241+0000 (oldest deadline 2024-05-03T20:53:31.740485+0000) 2024-05-03T20:53:46.107 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:45 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10[58033]: 2024-05-03T20:53:45.850+0000 7f48f1604700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.181:6822 osd.6 since back 2024-05-03T20:53:11.740981+0000 front 2024-05-03T20:53:11.740962+0000 (oldest deadline 2024-05-03T20:53:39.941596+0000) 2024-05-03T20:53:46.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:45 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:45.740+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:46.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:45 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:45.740+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:46.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:45 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:45.740+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:46.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:45 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:45.740+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:46.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:45 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:45.740+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:07.197190+0000 front 2024-05-03T20:53:07.197123+0000 (oldest deadline 2024-05-03T20:53:33.097275+0000) 2024-05-03T20:53:46.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:45 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:45.740+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6822 osd.6 since back 2024-05-03T20:53:13.097786+0000 front 2024-05-03T20:53:13.097805+0000 (oldest deadline 2024-05-03T20:53:36.597842+0000) 2024-05-03T20:53:46.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:45 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:45.740+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6820 osd.7 since back 2024-05-03T20:53:21.298968+0000 front 2024-05-03T20:53:21.298919+0000 (oldest deadline 2024-05-03T20:53:45.398773+0000) 2024-05-03T20:53:47.106 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:46 smithi196 podman[68986]: 2024-05-03 20:53:46.787496582 +0000 UTC m=+5.602375876 container died 3022140c2af4d805400196192815a66de86858d0ab20bc64ab6469f0b8c8fa8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240430) 2024-05-03T20:53:47.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:46 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:46.781+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:47.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:46 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:46.781+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:47.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:46 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:46.781+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:47.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:46 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:46.781+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:47.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:46 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:46.781+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:07.197190+0000 front 2024-05-03T20:53:07.197123+0000 (oldest deadline 2024-05-03T20:53:33.097275+0000) 2024-05-03T20:53:47.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:46 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:46.781+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6822 osd.6 since back 2024-05-03T20:53:13.097786+0000 front 2024-05-03T20:53:13.097805+0000 (oldest deadline 2024-05-03T20:53:36.597842+0000) 2024-05-03T20:53:47.115 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:46 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:46.781+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6820 osd.7 since back 2024-05-03T20:53:21.298968+0000 front 2024-05-03T20:53:21.298919+0000 (oldest deadline 2024-05-03T20:53:45.398773+0000) 2024-05-03T20:53:47.607 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:47 smithi196 podman[68986]: 2024-05-03 20:53:47.110133909 +0000 UTC m=+5.925013190 container cleanup 3022140c2af4d805400196192815a66de86858d0ab20bc64ab6469f0b8c8fa8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T20:53:47.607 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:47 smithi196 bash[68986]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10 2024-05-03T20:53:47.607 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:47 smithi196 podman[69003]: 2024-05-03 20:53:47.319503156 +0000 UTC m=+0.523191625 container remove 3022140c2af4d805400196192815a66de86858d0ab20bc64ab6469f0b8c8fa8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T20:53:47.929 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:47 smithi196 podman[69095]: 2024-05-03 20:53:47.645316444 +0000 UTC m=+0.113733913 container create 60f2cc8194574408fb4527b80bd769aa2e9080bcf1698ab7a9e00cacc2548047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-03T20:53:47.930 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:47 smithi196 podman[69095]: 2024-05-03 20:53:47.549449465 +0000 UTC m=+0.017866914 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T20:53:47.930 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:47 smithi196 podman[69095]: 2024-05-03 20:53:47.847608199 +0000 UTC m=+0.316025624 container init 60f2cc8194574408fb4527b80bd769aa2e9080bcf1698ab7a9e00cacc2548047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-05-03T20:53:47.930 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:47 smithi196 podman[69095]: 2024-05-03 20:53:47.850694325 +0000 UTC m=+0.319111761 container start 60f2cc8194574408fb4527b80bd769aa2e9080bcf1698ab7a9e00cacc2548047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-03T20:53:47.930 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:47 smithi196 podman[69095]: 2024-05-03 20:53:47.930744676 +0000 UTC m=+0.399162117 container attach 60f2cc8194574408fb4527b80bd769aa2e9080bcf1698ab7a9e00cacc2548047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-03T20:53:48.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:47 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:47.798+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:48.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:47 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:47.798+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:48.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:47 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:47.798+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:48.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:47 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:47.798+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:48.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:47 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:47.798+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:07.197190+0000 front 2024-05-03T20:53:07.197123+0000 (oldest deadline 2024-05-03T20:53:33.097275+0000) 2024-05-03T20:53:48.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:47 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:47.798+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6822 osd.6 since back 2024-05-03T20:53:13.097786+0000 front 2024-05-03T20:53:13.097805+0000 (oldest deadline 2024-05-03T20:53:36.597842+0000) 2024-05-03T20:53:48.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:47 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:47.798+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6820 osd.7 since back 2024-05-03T20:53:21.298968+0000 front 2024-05-03T20:53:21.298919+0000 (oldest deadline 2024-05-03T20:53:45.398773+0000) 2024-05-03T20:53:48.356 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:48 smithi196 podman[69095]: 2024-05-03 20:53:48.145991907 +0000 UTC m=+0.614409344 container died 60f2cc8194574408fb4527b80bd769aa2e9080bcf1698ab7a9e00cacc2548047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T20:53:48.720 INFO:journalctl@ceph.osd.10.smithi196.stdout:May 03 20:53:48 smithi196 podman[69121]: 2024-05-03 20:53:48.700687868 +0000 UTC m=+0.549711138 container remove 60f2cc8194574408fb4527b80bd769aa2e9080bcf1698ab7a9e00cacc2548047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T20:53:48.725 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.10.service' 2024-05-03T20:53:48.766 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:53:48.766 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-03T20:53:48.767 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-03T20:53:48.767 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.11 2024-05-03T20:53:49.103 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:48 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:48.753+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:49.103 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:48 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:48.753+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:49.104 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:48 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:48.753+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:49.104 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:48 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:48.753+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:49.104 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:48 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:48.753+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:07.197190+0000 front 2024-05-03T20:53:07.197123+0000 (oldest deadline 2024-05-03T20:53:33.097275+0000) 2024-05-03T20:53:49.104 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:48 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:48.753+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6822 osd.6 since back 2024-05-03T20:53:13.097786+0000 front 2024-05-03T20:53:13.097805+0000 (oldest deadline 2024-05-03T20:53:36.597842+0000) 2024-05-03T20:53:49.104 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:48 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:48.753+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6820 osd.7 since back 2024-05-03T20:53:21.298968+0000 front 2024-05-03T20:53:21.298919+0000 (oldest deadline 2024-05-03T20:53:45.398773+0000) 2024-05-03T20:53:49.104 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:48 smithi204 systemd[1]: Stopping Ceph osd.11 for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:53:49.104 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:49 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:49.102+0000 7f0c1665f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:53:49.104 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:49 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:49.102+0000 7f0c1665f700 -1 osd.11 86 *** Got signal Terminated *** 2024-05-03T20:53:49.105 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:49 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:49.102+0000 7f0c1665f700 -1 osd.11 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T20:53:50.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:49 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:49.721+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:50.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:49 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:49.721+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:50.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:49 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:49.721+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:50.115 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:49 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:49.721+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:50.115 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:49 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:49.721+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:07.197190+0000 front 2024-05-03T20:53:07.197123+0000 (oldest deadline 2024-05-03T20:53:33.097275+0000) 2024-05-03T20:53:50.115 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:49 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:49.721+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6822 osd.6 since back 2024-05-03T20:53:13.097786+0000 front 2024-05-03T20:53:13.097805+0000 (oldest deadline 2024-05-03T20:53:36.597842+0000) 2024-05-03T20:53:50.115 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:49 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:49.721+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6820 osd.7 since back 2024-05-03T20:53:21.298968+0000 front 2024-05-03T20:53:21.298919+0000 (oldest deadline 2024-05-03T20:53:45.398773+0000) 2024-05-03T20:53:51.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:50 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:50.707+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:51.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:50 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:50.707+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:51.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:50 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:50.707+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:51.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:50 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:50.707+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:51.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:50 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:50.707+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:07.197190+0000 front 2024-05-03T20:53:07.197123+0000 (oldest deadline 2024-05-03T20:53:33.097275+0000) 2024-05-03T20:53:51.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:50 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:50.707+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6822 osd.6 since back 2024-05-03T20:53:13.097786+0000 front 2024-05-03T20:53:13.097805+0000 (oldest deadline 2024-05-03T20:53:36.597842+0000) 2024-05-03T20:53:51.115 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:50 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:50.707+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6820 osd.7 since back 2024-05-03T20:53:21.298968+0000 front 2024-05-03T20:53:21.298919+0000 (oldest deadline 2024-05-03T20:53:45.398773+0000) 2024-05-03T20:53:52.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:51 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:51.707+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:52.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:51 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:51.707+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:52.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:51 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:51.707+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:52.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:51 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:51.707+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:52.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:51 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:51.707+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:07.197190+0000 front 2024-05-03T20:53:07.197123+0000 (oldest deadline 2024-05-03T20:53:33.097275+0000) 2024-05-03T20:53:52.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:51 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:51.707+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6822 osd.6 since back 2024-05-03T20:53:13.097786+0000 front 2024-05-03T20:53:13.097805+0000 (oldest deadline 2024-05-03T20:53:36.597842+0000) 2024-05-03T20:53:52.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:51 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:51.707+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6820 osd.7 since back 2024-05-03T20:53:21.298968+0000 front 2024-05-03T20:53:21.298919+0000 (oldest deadline 2024-05-03T20:53:45.398773+0000) 2024-05-03T20:53:53.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:52 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:52.703+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:53.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:52 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:52.703+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:53.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:52 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:52.703+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:53.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:52 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:52.703+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:53.115 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:52 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:52.703+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:07.197190+0000 front 2024-05-03T20:53:07.197123+0000 (oldest deadline 2024-05-03T20:53:33.097275+0000) 2024-05-03T20:53:53.115 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:52 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:52.703+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6822 osd.6 since back 2024-05-03T20:53:13.097786+0000 front 2024-05-03T20:53:13.097805+0000 (oldest deadline 2024-05-03T20:53:36.597842+0000) 2024-05-03T20:53:53.115 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:52 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:52.703+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6820 osd.7 since back 2024-05-03T20:53:21.298968+0000 front 2024-05-03T20:53:21.298919+0000 (oldest deadline 2024-05-03T20:53:45.398773+0000) 2024-05-03T20:53:54.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:53 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:53.656+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6806 osd.0 since back 2024-05-03T20:52:26.891304+0000 front 2024-05-03T20:52:26.891323+0000 (oldest deadline 2024-05-03T20:52:49.791312+0000) 2024-05-03T20:53:54.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:53 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:53.656+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6804 osd.1 since back 2024-05-03T20:52:34.492511+0000 front 2024-05-03T20:52:34.492713+0000 (oldest deadline 2024-05-03T20:53:00.392560+0000) 2024-05-03T20:53:54.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:53 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:53.656+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-05-03T20:52:40.392993+0000 front 2024-05-03T20:52:40.393054+0000 (oldest deadline 2024-05-03T20:53:04.493119+0000) 2024-05-03T20:53:54.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:53 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:53.656+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6812 osd.4 since back 2024-05-03T20:52:59.596408+0000 front 2024-05-03T20:52:59.596595+0000 (oldest deadline 2024-05-03T20:53:24.296367+0000) 2024-05-03T20:53:54.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:53 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:53.656+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-05-03T20:53:07.197190+0000 front 2024-05-03T20:53:07.197123+0000 (oldest deadline 2024-05-03T20:53:33.097275+0000) 2024-05-03T20:53:54.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:53 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:53.656+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.181:6822 osd.6 since back 2024-05-03T20:53:13.097786+0000 front 2024-05-03T20:53:13.097805+0000 (oldest deadline 2024-05-03T20:53:36.597842+0000) 2024-05-03T20:53:54.114 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:53 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11[58984]: 2024-05-03T20:53:53.656+0000 7f0c12c53700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.196:6820 osd.7 since back 2024-05-03T20:53:21.298968+0000 front 2024-05-03T20:53:21.298919+0000 (oldest deadline 2024-05-03T20:53:45.398773+0000) 2024-05-03T20:53:54.820 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:54 smithi204 podman[69998]: 2024-05-03 20:53:54.488069839 +0000 UTC m=+5.506326363 container died be95d51f72ca05e2d89df17f3b34401d27967b497006f803eb30bdabd95821a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-03T20:53:55.095 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:54 smithi204 podman[69998]: 2024-05-03 20:53:54.921040628 +0000 UTC m=+5.939297147 container cleanup be95d51f72ca05e2d89df17f3b34401d27967b497006f803eb30bdabd95821a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-03T20:53:55.096 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:54 smithi204 bash[69998]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11 2024-05-03T20:53:55.363 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:55 smithi204 podman[70014]: 2024-05-03 20:53:55.097163613 +0000 UTC m=+0.603432338 container remove be95d51f72ca05e2d89df17f3b34401d27967b497006f803eb30bdabd95821a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-03T20:53:55.832 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:55 smithi204 podman[70103]: 2024-05-03 20:53:55.443063556 +0000 UTC m=+0.018144927 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T20:53:55.833 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:55 smithi204 podman[70103]: 2024-05-03 20:53:55.548055137 +0000 UTC m=+0.123136479 container create c6cfb7443bd98129a252805b2cc6495188c305816ddaa75e226bea2ba62bf56c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-03T20:53:55.833 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:55 smithi204 podman[70103]: 2024-05-03 20:53:55.776839638 +0000 UTC m=+0.351920985 container init c6cfb7443bd98129a252805b2cc6495188c305816ddaa75e226bea2ba62bf56c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T20:53:55.833 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:55 smithi204 podman[70103]: 2024-05-03 20:53:55.780089005 +0000 UTC m=+0.355170347 container start c6cfb7443bd98129a252805b2cc6495188c305816ddaa75e226bea2ba62bf56c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-03T20:53:55.833 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:55 smithi204 podman[70103]: 2024-05-03 20:53:55.834331548 +0000 UTC m=+0.409412904 container attach c6cfb7443bd98129a252805b2cc6495188c305816ddaa75e226bea2ba62bf56c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-03T20:53:56.113 INFO:journalctl@ceph.osd.11.smithi204.stdout:May 03 20:53:56 smithi204 podman[70103]: 2024-05-03 20:53:56.076683763 +0000 UTC m=+0.651765123 container died c6cfb7443bd98129a252805b2cc6495188c305816ddaa75e226bea2ba62bf56c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-03T20:53:56.637 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@osd.11.service' 2024-05-03T20:53:56.674 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:53:56.674 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-03T20:53:56.675 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-03T20:53:56.675 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.a 2024-05-03T20:54:01.428 INFO:journalctl@ceph.mds.a.smithi181.stdout:May 03 20:54:01 smithi181 podman[92318]: 2024-05-03 20:54:01.064435645 +0000 UTC m=+4.194801967 container died 0743a3767730f8fe78c1e8e2d17c8c843d45afcfe5851bdaa8e109e6edb47d0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-03T20:54:01.696 INFO:journalctl@ceph.mds.a.smithi181.stdout:May 03 20:54:01 smithi181 podman[92318]: 2024-05-03 20:54:01.512516014 +0000 UTC m=+4.642882331 container cleanup 0743a3767730f8fe78c1e8e2d17c8c843d45afcfe5851bdaa8e109e6edb47d0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-a, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-03T20:54:01.696 INFO:journalctl@ceph.mds.a.smithi181.stdout:May 03 20:54:01 smithi181 bash[92318]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-a 2024-05-03T20:54:01.927 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.a.service' 2024-05-03T20:54:01.948 INFO:journalctl@ceph.mds.a.smithi181.stdout:May 03 20:54:01 smithi181 podman[92343]: 2024-05-03 20:54:01.697074817 +0000 UTC m=+0.629943464 container remove 0743a3767730f8fe78c1e8e2d17c8c843d45afcfe5851bdaa8e109e6edb47d0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-03T20:54:01.949 INFO:journalctl@ceph.mds.a.smithi181.stdout:May 03 20:54:01 smithi181 systemd[1]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.a.service: Deactivated successfully. 2024-05-03T20:54:01.949 INFO:journalctl@ceph.mds.a.smithi181.stdout:May 03 20:54:01 smithi181 systemd[1]: Stopped Ceph mds.a for 1cd058ee-098b-11ef-bc97-c7b262605968. 2024-05-03T20:54:01.949 INFO:journalctl@ceph.mds.a.smithi181.stdout:May 03 20:54:01 smithi181 systemd[1]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.a.service: Consumed 2.560s CPU time. 2024-05-03T20:54:01.962 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:54:01.962 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-03T20:54:01.963 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-03T20:54:01.963 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.d 2024-05-03T20:54:02.213 INFO:journalctl@ceph.mds.d.smithi181.stdout:May 03 20:54:02 smithi181 systemd[1]: Stopping Ceph mds.d for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:54:02.713 INFO:journalctl@ceph.mds.d.smithi181.stdout:May 03 20:54:02 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-d[71077]: 2024-05-03T20:54:02.341+0000 7fe6aaeb4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:54:02.713 INFO:journalctl@ceph.mds.d.smithi181.stdout:May 03 20:54:02 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-d[71077]: 2024-05-03T20:54:02.341+0000 7fe6aaeb4700 -1 mds.d *** got signal Terminated *** 2024-05-03T20:54:05.340 INFO:journalctl@ceph.mds.d.smithi181.stdout:May 03 20:54:04 smithi181 podman[92451]: 2024-05-03 20:54:04.988603304 +0000 UTC m=+2.808582018 container died 50f687277cd50cab81ca99e1cbda2c138a0c0c74e99486af6f5a51506ac7b6d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T20:54:05.713 INFO:journalctl@ceph.mds.d.smithi181.stdout:May 03 20:54:05 smithi181 podman[92451]: 2024-05-03 20:54:05.433145145 +0000 UTC m=+3.253123867 container cleanup 50f687277cd50cab81ca99e1cbda2c138a0c0c74e99486af6f5a51506ac7b6d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-d, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-03T20:54:05.713 INFO:journalctl@ceph.mds.d.smithi181.stdout:May 03 20:54:05 smithi181 bash[92451]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-d 2024-05-03T20:54:05.850 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.d.service' 2024-05-03T20:54:05.881 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:54:05.882 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-03T20:54:05.882 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-03T20:54:05.882 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.g 2024-05-03T20:54:06.202 INFO:journalctl@ceph.mds.g.smithi181.stdout:May 03 20:54:05 smithi181 systemd[1]: Stopping Ceph mds.g for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:54:06.463 INFO:journalctl@ceph.mds.g.smithi181.stdout:May 03 20:54:06 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-g[71911]: 2024-05-03T20:54:06.202+0000 7fd58b016700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:54:06.463 INFO:journalctl@ceph.mds.g.smithi181.stdout:May 03 20:54:06 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-g[71911]: 2024-05-03T20:54:06.202+0000 7fd58b016700 -1 mds.g *** got signal Terminated *** 2024-05-03T20:54:09.911 INFO:journalctl@ceph.mds.g.smithi181.stdout:May 03 20:54:09 smithi181 podman[92571]: 2024-05-03 20:54:09.648236697 +0000 UTC m=+3.544790777 container died d37b47955bc7e24e67ee3370ccde96dbd9fc95958c755eaa4e675fc0fc2fe912 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-g, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T20:54:10.213 INFO:journalctl@ceph.mds.g.smithi181.stdout:May 03 20:54:10 smithi181 podman[92571]: 2024-05-03 20:54:10.12849039 +0000 UTC m=+4.025044470 container cleanup d37b47955bc7e24e67ee3370ccde96dbd9fc95958c755eaa4e675fc0fc2fe912 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-g, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-03T20:54:10.213 INFO:journalctl@ceph.mds.g.smithi181.stdout:May 03 20:54:10 smithi181 bash[92571]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-g 2024-05-03T20:54:10.508 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.g.service' 2024-05-03T20:54:10.539 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:54:10.540 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-03T20:54:10.540 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-03T20:54:10.540 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.j 2024-05-03T20:54:10.713 INFO:journalctl@ceph.mds.j.smithi181.stdout:May 03 20:54:10 smithi181 systemd[1]: Stopping Ceph mds.j for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:54:11.213 INFO:journalctl@ceph.mds.j.smithi181.stdout:May 03 20:54:10 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-j[70645]: 2024-05-03T20:54:10.864+0000 7fa6a5f55700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:54:11.213 INFO:journalctl@ceph.mds.j.smithi181.stdout:May 03 20:54:10 smithi181 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-j[70645]: 2024-05-03T20:54:10.864+0000 7fa6a5f55700 -1 mds.j *** got signal Terminated *** 2024-05-03T20:54:13.281 INFO:journalctl@ceph.mds.j.smithi181.stdout:May 03 20:54:12 smithi181 podman[92691]: 2024-05-03 20:54:12.981119961 +0000 UTC m=+2.227986622 container died 37665f477d8460617beb522bc4914e37884ba9b73b0a7d81265eda0c1351c115 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-j, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-03T20:54:13.598 INFO:journalctl@ceph.mds.j.smithi181.stdout:May 03 20:54:13 smithi181 podman[92691]: 2024-05-03 20:54:13.349139762 +0000 UTC m=+2.596006473 container cleanup 37665f477d8460617beb522bc4914e37884ba9b73b0a7d81265eda0c1351c115 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-03T20:54:13.757 INFO:journalctl@ceph.mds.j.smithi181.stdout:May 03 20:54:13 smithi181 bash[92691]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-j 2024-05-03T20:54:13.758 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.j.service' 2024-05-03T20:54:13.789 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:54:13.790 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-03T20:54:13.790 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-03T20:54:13.790 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.b 2024-05-03T20:54:14.073 INFO:journalctl@ceph.mds.b.smithi196.stdout:May 03 20:54:13 smithi196 systemd[1]: Stopping Ceph mds.b for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:54:14.330 INFO:journalctl@ceph.mds.b.smithi196.stdout:May 03 20:54:14 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-b[63557]: 2024-05-03T20:54:14.074+0000 7fcf08e69700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:54:14.330 INFO:journalctl@ceph.mds.b.smithi196.stdout:May 03 20:54:14 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-b[63557]: 2024-05-03T20:54:14.074+0000 7fcf08e69700 -1 mds.b *** got signal Terminated *** 2024-05-03T20:54:17.932 INFO:journalctl@ceph.mds.b.smithi196.stdout:May 03 20:54:17 smithi196 podman[69199]: 2024-05-03 20:54:17.65884751 +0000 UTC m=+3.678108637 container died 0cc7fb0361cde5de70797ee63dcdc4ae8ced119b4eb0ef34fbd7df2369eca36a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-03T20:54:18.290 INFO:journalctl@ceph.mds.b.smithi196.stdout:May 03 20:54:18 smithi196 podman[69199]: 2024-05-03 20:54:18.033054696 +0000 UTC m=+4.052315832 container cleanup 0cc7fb0361cde5de70797ee63dcdc4ae8ced119b4eb0ef34fbd7df2369eca36a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=) 2024-05-03T20:54:18.291 INFO:journalctl@ceph.mds.b.smithi196.stdout:May 03 20:54:18 smithi196 bash[69199]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-b 2024-05-03T20:54:18.454 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.b.service' 2024-05-03T20:54:18.487 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:54:18.487 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-03T20:54:18.488 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-03T20:54:18.488 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.e 2024-05-03T20:54:18.810 INFO:journalctl@ceph.mds.e.smithi196.stdout:May 03 20:54:18 smithi196 systemd[1]: Stopping Ceph mds.e for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:54:19.106 INFO:journalctl@ceph.mds.e.smithi196.stdout:May 03 20:54:18 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-e[62924]: 2024-05-03T20:54:18.811+0000 7f126d460700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:54:19.107 INFO:journalctl@ceph.mds.e.smithi196.stdout:May 03 20:54:18 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-e[62924]: 2024-05-03T20:54:18.811+0000 7f126d460700 -1 mds.e *** got signal Terminated *** 2024-05-03T20:54:21.968 INFO:journalctl@ceph.mds.e.smithi196.stdout:May 03 20:54:21 smithi196 podman[69331]: 2024-05-03 20:54:21.658027037 +0000 UTC m=+2.954245113 container died cb61e4c3e20c5a4f57c4ce075d40df901711176ef3ab2fea960b97d546b327af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T20:54:22.356 INFO:journalctl@ceph.mds.e.smithi196.stdout:May 03 20:54:21 smithi196 podman[69331]: 2024-05-03 20:54:21.96952651 +0000 UTC m=+3.265744597 container cleanup cb61e4c3e20c5a4f57c4ce075d40df901711176ef3ab2fea960b97d546b327af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS) 2024-05-03T20:54:22.357 INFO:journalctl@ceph.mds.e.smithi196.stdout:May 03 20:54:21 smithi196 bash[69331]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-e 2024-05-03T20:54:22.357 INFO:journalctl@ceph.mds.e.smithi196.stdout:May 03 20:54:22 smithi196 podman[69345]: 2024-05-03 20:54:22.153819309 +0000 UTC m=+0.493862805 container remove cb61e4c3e20c5a4f57c4ce075d40df901711176ef3ab2fea960b97d546b327af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD) 2024-05-03T20:54:22.399 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.e.service' 2024-05-03T20:54:22.432 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:54:22.432 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-03T20:54:22.432 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-03T20:54:22.433 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.h 2024-05-03T20:54:22.738 INFO:journalctl@ceph.mds.h.smithi196.stdout:May 03 20:54:22 smithi196 systemd[1]: Stopping Ceph mds.h for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:54:23.106 INFO:journalctl@ceph.mds.h.smithi196.stdout:May 03 20:54:22 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-h[63236]: 2024-05-03T20:54:22.738+0000 7f6869979700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:54:23.107 INFO:journalctl@ceph.mds.h.smithi196.stdout:May 03 20:54:22 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-h[63236]: 2024-05-03T20:54:22.738+0000 7f6869979700 -1 mds.h *** got signal Terminated *** 2024-05-03T20:54:25.937 INFO:journalctl@ceph.mds.h.smithi196.stdout:May 03 20:54:25 smithi196 podman[69451]: 2024-05-03 20:54:25.661813188 +0000 UTC m=+3.010783678 container died 2f6a0b1b3e94588ec9c390caa51b67475fce66d4330833638def031dee65b837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-h, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430) 2024-05-03T20:54:25.938 INFO:journalctl@ceph.mds.h.smithi196.stdout:May 03 20:54:25 smithi196 podman[69451]: 2024-05-03 20:54:25.93892165 +0000 UTC m=+3.287892137 container cleanup 2f6a0b1b3e94588ec9c390caa51b67475fce66d4330833638def031dee65b837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-h, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, CEPH_POINT_RELEASE=) 2024-05-03T20:54:26.196 INFO:journalctl@ceph.mds.h.smithi196.stdout:May 03 20:54:25 smithi196 bash[69451]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-h 2024-05-03T20:54:26.315 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.h.service' 2024-05-03T20:54:26.345 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:54:26.345 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-03T20:54:26.345 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-03T20:54:26.345 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.k 2024-05-03T20:54:26.606 INFO:journalctl@ceph.mds.k.smithi196.stdout:May 03 20:54:26 smithi196 systemd[1]: Stopping Ceph mds.k for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:54:27.107 INFO:journalctl@ceph.mds.k.smithi196.stdout:May 03 20:54:26 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-k[62601]: 2024-05-03T20:54:26.658+0000 7ff40f969700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:54:27.107 INFO:journalctl@ceph.mds.k.smithi196.stdout:May 03 20:54:26 smithi196 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-k[62601]: 2024-05-03T20:54:26.658+0000 7ff40f969700 -1 mds.k *** got signal Terminated *** 2024-05-03T20:54:29.974 INFO:journalctl@ceph.mds.k.smithi196.stdout:May 03 20:54:29 smithi196 podman[69571]: 2024-05-03 20:54:29.660941494 +0000 UTC m=+3.106385215 container died dcb17aadb68d2050a88c5370bb150af6f345cf8ebb100d8deea7c2acb6c80c52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-k, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-03T20:54:29.974 INFO:journalctl@ceph.mds.k.smithi196.stdout:May 03 20:54:29 smithi196 podman[69571]: 2024-05-03 20:54:29.975449977 +0000 UTC m=+3.420893683 container cleanup dcb17aadb68d2050a88c5370bb150af6f345cf8ebb100d8deea7c2acb6c80c52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-k, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-03T20:54:30.250 INFO:journalctl@ceph.mds.k.smithi196.stdout:May 03 20:54:29 smithi196 bash[69571]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-k 2024-05-03T20:54:30.250 INFO:journalctl@ceph.mds.k.smithi196.stdout:May 03 20:54:30 smithi196 podman[69585]: 2024-05-03 20:54:30.14329598 +0000 UTC m=+0.480343916 container remove dcb17aadb68d2050a88c5370bb150af6f345cf8ebb100d8deea7c2acb6c80c52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T20:54:30.260 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.k.service' 2024-05-03T20:54:30.290 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:54:30.291 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-03T20:54:30.291 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-03T20:54:30.291 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.c 2024-05-03T20:54:30.595 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 03 20:54:30 smithi204 systemd[1]: Stopping Ceph mds.c for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:54:30.857 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 03 20:54:30 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-c[64248]: 2024-05-03T20:54:30.595+0000 7f7796f62700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:54:30.857 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 03 20:54:30 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-c[64248]: 2024-05-03T20:54:30.595+0000 7f7796f62700 -1 mds.c *** got signal Terminated *** 2024-05-03T20:54:33.987 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 03 20:54:33 smithi204 podman[70209]: 2024-05-03 20:54:33.66447231 +0000 UTC m=+3.180377863 container died df2ea6b1690e1992ee18250f1f41446524f127dee49b85d7e21726e4ae7ffc56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, ceph=True) 2024-05-03T20:54:34.295 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 03 20:54:34 smithi204 podman[70209]: 2024-05-03 20:54:34.053541964 +0000 UTC m=+3.569447521 container cleanup df2ea6b1690e1992ee18250f1f41446524f127dee49b85d7e21726e4ae7ffc56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T20:54:34.295 INFO:journalctl@ceph.mds.c.smithi204.stdout:May 03 20:54:34 smithi204 bash[70209]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-c 2024-05-03T20:54:34.433 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.c.service' 2024-05-03T20:54:34.469 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:54:34.469 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-03T20:54:34.469 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-03T20:54:34.469 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.f 2024-05-03T20:54:34.863 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 03 20:54:34 smithi204 systemd[1]: Stopping Ceph mds.f for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:54:34.863 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 03 20:54:34 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-f[63943]: 2024-05-03T20:54:34.782+0000 7ff6052e9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:54:34.864 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 03 20:54:34 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-f[63943]: 2024-05-03T20:54:34.782+0000 7ff6052e9700 -1 mds.f *** got signal Terminated *** 2024-05-03T20:54:37.907 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 03 20:54:37 smithi204 podman[70340]: 2024-05-03 20:54:37.651534836 +0000 UTC m=+2.964909445 container died a4aca1db7f9da0393c9ed052931f4d118e629e43dcb57d7b48eb6c57a8a3e947 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, ceph=True, org.label-schema.schema-version=1.0) 2024-05-03T20:54:38.274 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 03 20:54:38 smithi204 podman[70340]: 2024-05-03 20:54:38.007332367 +0000 UTC m=+3.320706976 container cleanup a4aca1db7f9da0393c9ed052931f4d118e629e43dcb57d7b48eb6c57a8a3e947 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-f, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-03T20:54:38.274 INFO:journalctl@ceph.mds.f.smithi204.stdout:May 03 20:54:38 smithi204 bash[70340]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-f 2024-05-03T20:54:38.387 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.f.service' 2024-05-03T20:54:38.420 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:54:38.420 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-03T20:54:38.421 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-03T20:54:38.421 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.i 2024-05-03T20:54:38.613 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 03 20:54:38 smithi204 systemd[1]: Stopping Ceph mds.i for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:54:39.113 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 03 20:54:38 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-i[64570]: 2024-05-03T20:54:38.727+0000 7fdeab68c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:54:39.113 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 03 20:54:38 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-i[64570]: 2024-05-03T20:54:38.727+0000 7fdeab68c700 -1 mds.i *** got signal Terminated *** 2024-05-03T20:54:41.919 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 03 20:54:41 smithi204 podman[70462]: 2024-05-03 20:54:41.648812007 +0000 UTC m=+3.017247677 container died 23106480b45ab8d258d63bec65bfa2e99c7e3cfbfbd88cff081f693a96d6bac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T20:54:42.269 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 03 20:54:42 smithi204 podman[70462]: 2024-05-03 20:54:42.019723763 +0000 UTC m=+3.388159429 container cleanup 23106480b45ab8d258d63bec65bfa2e99c7e3cfbfbd88cff081f693a96d6bac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-i, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T20:54:42.269 INFO:journalctl@ceph.mds.i.smithi204.stdout:May 03 20:54:42 smithi204 bash[70462]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-i 2024-05-03T20:54:42.391 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.i.service' 2024-05-03T20:54:42.422 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:54:42.422 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-03T20:54:42.423 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-03T20:54:42.423 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.l 2024-05-03T20:54:42.613 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 03 20:54:42 smithi204 systemd[1]: Stopping Ceph mds.l for 1cd058ee-098b-11ef-bc97-c7b262605968... 2024-05-03T20:54:43.114 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 03 20:54:42 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-l[63631]: 2024-05-03T20:54:42.747+0000 7f17686f8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T20:54:43.114 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 03 20:54:42 smithi204 ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-l[63631]: 2024-05-03T20:54:42.747+0000 7f17686f8700 -1 mds.l *** got signal Terminated *** 2024-05-03T20:54:46.006 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 03 20:54:45 smithi204 podman[70581]: 2024-05-03 20:54:45.667410396 +0000 UTC m=+3.028796914 container died 271a1961e50251175d5ef9bcea4e0e207fe041be3dfa74612cedf5d95173c8c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-l, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2) 2024-05-03T20:54:46.300 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 03 20:54:46 smithi204 podman[70581]: 2024-05-03 20:54:46.089923354 +0000 UTC m=+3.451309870 container cleanup 271a1961e50251175d5ef9bcea4e0e207fe041be3dfa74612cedf5d95173c8c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-03T20:54:46.300 INFO:journalctl@ceph.mds.l.smithi204.stdout:May 03 20:54:46 smithi204 bash[70581]: ceph-1cd058ee-098b-11ef-bc97-c7b262605968-mds-l 2024-05-03T20:54:46.519 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cd058ee-098b-11ef-bc97-c7b262605968@mds.l.service' 2024-05-03T20:54:46.549 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T20:54:46.549 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-03T20:54:46.549 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-03T20:54:46.740 INFO:teuthology.orchestra.run.smithi181.stdout:Deleting cluster with fsid: 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:54:49.160 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-03T20:54:49.348 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:54:51.791 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-03T20:54:51.984 INFO:teuthology.orchestra.run.smithi204.stdout:Deleting cluster with fsid: 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:54:54.267 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T20:54:54.299 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T20:54:54.330 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T20:54:54.355 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-03T20:54:54.357 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687487/remote/smithi181/crash 2024-05-03T20:54:54.358 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/crash -- . 2024-05-03T20:54:54.396 INFO:teuthology.orchestra.run.smithi181.stderr:tar: /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-03T20:54:54.396 INFO:teuthology.orchestra.run.smithi181.stderr:tar: Error is not recoverable: exiting now 2024-05-03T20:54:54.398 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687487/remote/smithi196/crash 2024-05-03T20:54:54.399 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/crash -- . 2024-05-03T20:54:54.437 INFO:teuthology.orchestra.run.smithi196.stderr:tar: /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-03T20:54:54.437 INFO:teuthology.orchestra.run.smithi196.stderr:tar: Error is not recoverable: exiting now 2024-05-03T20:54:54.439 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687487/remote/smithi204/crash 2024-05-03T20:54:54.439 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/crash -- . 2024-05-03T20:54:54.485 INFO:teuthology.orchestra.run.smithi204.stderr:tar: /var/lib/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-03T20:54:54.485 INFO:teuthology.orchestra.run.smithi204.stderr:tar: Error is not recoverable: exiting now 2024-05-03T20:54:54.487 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-03T20:54:54.487 DEBUG:teuthology.orchestra.run.smithi181:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-05-03T20:54:54.519 INFO:tasks.cephadm:Compressing logs... 2024-05-03T20:54:54.519 DEBUG:teuthology.orchestra.run.smithi181:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-03T20:54:54.562 DEBUG:teuthology.orchestra.run.smithi196:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-03T20:54:54.565 DEBUG:teuthology.orchestra.run.smithi204:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-03T20:54:54.586 INFO:teuthology.orchestra.run.smithi181.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.log 2024-05-03T20:54:54.586 INFO:teuthology.orchestra.run.smithi181.stderr:: No such file or directory 2024-05-03T20:54:54.586 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-03T20:54:54.586 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.g.log 2024-05-03T20:54:54.587 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.j.log 2024-05-03T20:54:54.589 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.3.log 2024-05-03T20:54:54.589 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.0.log 2024-05-03T20:54:54.589 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.j.log: /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mgr.x.log 2024-05-03T20:54:54.590 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mon.a.log 2024-05-03T20:54:54.590 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.6.log 2024-05-03T20:54:54.590 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-03T20:54:54.590 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mon.a.log: 90.4% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-03T20:54:54.596 INFO:teuthology.orchestra.run.smithi181.stderr: 94.7% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.j.log.gz 2024-05-03T20:54:54.603 INFO:teuthology.orchestra.run.smithi196.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.log 2024-05-03T20:54:54.603 INFO:teuthology.orchestra.run.smithi196.stderr:: No such file or directory 2024-05-03T20:54:54.604 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-03T20:54:54.604 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.7.log 2024-05-03T20:54:54.604 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.log: /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.b.log 2024-05-03T20:54:54.604 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.4.log 2024-05-03T20:54:54.604 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.1.log 2024-05-03T20:54:54.604 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.b.log: /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mgr.y.log 2024-05-03T20:54:54.604 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.h.log 2024-05-03T20:54:54.604 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.10.log 2024-05-03T20:54:54.605 INFO:teuthology.orchestra.run.smithi196.stderr: 90.6% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-03T20:54:54.605 INFO:teuthology.orchestra.run.smithi196.stderr: 89.1% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-03T20:54:54.605 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-03T20:54:54.605 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.e.log 2024-05-03T20:54:54.605 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.cephadm.log: 82.3% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-03T20:54:54.605 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-03T20:54:54.605 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.e.log: 93.1% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mgr.y.log.gz 2024-05-03T20:54:54.607 INFO:teuthology.orchestra.run.smithi204.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.log 2024-05-03T20:54:54.607 INFO:teuthology.orchestra.run.smithi204.stderr:: No such file or directory 2024-05-03T20:54:54.607 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.2.log 2024-05-03T20:54:54.608 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.f.log 2024-05-03T20:54:54.608 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.log: /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-03T20:54:54.608 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.5.log 2024-05-03T20:54:54.608 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.c.log 2024-05-03T20:54:54.608 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.audit.log: /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mon.c.log 2024-05-03T20:54:54.608 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mgr.z.log 2024-05-03T20:54:54.608 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.i.log 2024-05-03T20:54:54.608 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mgr.z.log: 89.1% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-03T20:54:54.609 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-03T20:54:54.609 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.i.log: 90.6% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-03T20:54:54.609 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.11.log 2024-05-03T20:54:54.609 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.cephadm.log: 82.3% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-03T20:54:54.609 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.8.log 2024-05-03T20:54:54.609 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.11.log: 93.3% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mgr.z.log.gz 2024-05-03T20:54:54.609 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-03T20:54:54.609 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-03T20:54:54.610 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.a.log 2024-05-03T20:54:54.610 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-03T20:54:54.610 INFO:teuthology.orchestra.run.smithi181.stderr: 82.8% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-03T20:54:54.612 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.d.log 2024-05-03T20:54:54.617 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mon.b.log 2024-05-03T20:54:54.624 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.9.log 2024-05-03T20:54:54.627 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.d.log: 91.1% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-03T20:54:54.628 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-03T20:54:54.630 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.l.log 2024-05-03T20:54:54.630 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.k.log 2024-05-03T20:54:54.631 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-03T20:54:54.635 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.9.log: 94.6% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.d.log.gz 2024-05-03T20:54:54.640 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-03T20:54:54.643 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-03T20:54:54.648 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.l.log: 94.2% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.c.log.gz 2024-05-03T20:54:54.648 INFO:teuthology.orchestra.run.smithi204.stderr: 94.1% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.f.log.gz 2024-05-03T20:54:54.649 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/cephadm.log: 91.0% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-03T20:54:54.651 INFO:teuthology.orchestra.run.smithi204.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-03T20:54:54.655 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.k.log: 94.0% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.h.log.gz 2024-05-03T20:54:54.657 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-03T20:54:54.661 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/cephadm.log: 93.7% 94.7% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.g.log.gz 2024-05-03T20:54:54.661 INFO:teuthology.orchestra.run.smithi181.stderr: -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-03T20:54:54.663 INFO:teuthology.orchestra.run.smithi204.stderr: 93.9% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.i.log.gz 2024-05-03T20:54:54.681 INFO:teuthology.orchestra.run.smithi181.stderr: 94.7% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.a.log.gz 2024-05-03T20:54:54.688 INFO:teuthology.orchestra.run.smithi196.stderr: 94.5% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.e.log.gz 2024-05-03T20:54:54.700 INFO:teuthology.orchestra.run.smithi204.stderr: 94.7% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.l.log.gz 2024-05-03T20:54:54.731 INFO:teuthology.orchestra.run.smithi196.stderr: 94.6% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.k.log.gz 2024-05-03T20:54:54.835 INFO:teuthology.orchestra.run.smithi196.stderr: 94.0% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mds.b.log.gz 2024-05-03T20:54:54.921 INFO:teuthology.orchestra.run.smithi181.stderr: 90.3% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mgr.x.log.gz 2024-05-03T20:54:54.941 INFO:teuthology.orchestra.run.smithi204.stderr: 92.0% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mon.c.log.gz 2024-05-03T20:54:54.986 INFO:teuthology.orchestra.run.smithi196.stderr: 92.2% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mon.b.log.gz 2024-05-03T20:54:55.105 INFO:teuthology.orchestra.run.smithi204.stderr: 92.4% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.8.log.gz 2024-05-03T20:54:55.135 INFO:teuthology.orchestra.run.smithi204.stderr: 92.6% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.11.log.gz 2024-05-03T20:54:55.189 INFO:teuthology.orchestra.run.smithi181.stderr: 92.3% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.6.log.gz 2024-05-03T20:54:55.215 INFO:teuthology.orchestra.run.smithi204.stderr: 92.3% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.2.log.gz 2024-05-03T20:54:55.228 INFO:teuthology.orchestra.run.smithi196.stderr: 92.5% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.1.log.gz 2024-05-03T20:54:55.267 INFO:teuthology.orchestra.run.smithi181.stderr: 92.0% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.9.log.gz 2024-05-03T20:54:55.269 INFO:teuthology.orchestra.run.smithi181.stderr: 92.5% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.0.log.gz 2024-05-03T20:54:55.290 INFO:teuthology.orchestra.run.smithi181.stderr: 92.5% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.3.log.gz 2024-05-03T20:54:55.291 INFO:teuthology.orchestra.run.smithi196.stderr: 92.3% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.4.log.gz 2024-05-03T20:54:55.319 INFO:teuthology.orchestra.run.smithi196.stderr: 92.3% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.7.log.gz 2024-05-03T20:54:55.374 INFO:teuthology.orchestra.run.smithi204.stderr: 92.4% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.5.log.gz 2024-05-03T20:54:55.376 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-05-03T20:54:55.376 INFO:teuthology.orchestra.run.smithi204.stderr:real 0m0.797s 2024-05-03T20:54:55.377 INFO:teuthology.orchestra.run.smithi204.stderr:user 0m2.954s 2024-05-03T20:54:55.377 INFO:teuthology.orchestra.run.smithi204.stderr:sys 0m0.133s 2024-05-03T20:54:55.461 INFO:teuthology.orchestra.run.smithi196.stderr: 92.7% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-osd.10.log.gz 2024-05-03T20:54:55.463 INFO:teuthology.orchestra.run.smithi196.stderr: 2024-05-03T20:54:55.463 INFO:teuthology.orchestra.run.smithi196.stderr:real 0m0.883s 2024-05-03T20:54:55.463 INFO:teuthology.orchestra.run.smithi196.stderr:user 0m3.646s 2024-05-03T20:54:55.463 INFO:teuthology.orchestra.run.smithi196.stderr:sys 0m0.147s 2024-05-03T20:54:55.503 INFO:teuthology.orchestra.run.smithi181.stderr: 90.4% -- replaced with /var/log/ceph/1cd058ee-098b-11ef-bc97-c7b262605968/ceph-mon.a.log.gz 2024-05-03T20:54:55.505 INFO:teuthology.orchestra.run.smithi181.stderr: 2024-05-03T20:54:55.505 INFO:teuthology.orchestra.run.smithi181.stderr:real 0m0.929s 2024-05-03T20:54:55.505 INFO:teuthology.orchestra.run.smithi181.stderr:user 0m3.920s 2024-05-03T20:54:55.505 INFO:teuthology.orchestra.run.smithi181.stderr:sys 0m0.144s 2024-05-03T20:54:55.505 INFO:tasks.cephadm:Archiving logs... 2024-05-03T20:54:55.506 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687487/remote/smithi181/log 2024-05-03T20:54:55.507 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-03T20:54:56.136 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687487/remote/smithi196/log 2024-05-03T20:54:56.137 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-03T20:54:56.696 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687487/remote/smithi204/log 2024-05-03T20:54:56.697 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-03T20:54:57.149 INFO:tasks.cephadm:Removing cluster... 2024-05-03T20:54:57.149 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 --force 2024-05-03T20:54:57.341 INFO:teuthology.orchestra.run.smithi181.stdout:Deleting cluster with fsid: 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:54:57.601 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 --force 2024-05-03T20:54:57.793 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:54:58.041 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1cd058ee-098b-11ef-bc97-c7b262605968 --force 2024-05-03T20:54:58.231 INFO:teuthology.orchestra.run.smithi204.stdout:Deleting cluster with fsid: 1cd058ee-098b-11ef-bc97-c7b262605968 2024-05-03T20:54:58.662 INFO:tasks.cephadm:Removing cephadm ... 2024-05-03T20:54:58.663 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-03T20:54:58.678 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-03T20:54:58.695 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-03T20:54:58.710 INFO:tasks.cephadm:Teardown complete 2024-05-03T20:54:58.711 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-03T20:54:58.721 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-03T20:54:58.721 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-03T20:54:58.724 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-03T20:54:58.737 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-03T20:54:58.841 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-03T20:54:58.850 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-03T20:54:58.851 DEBUG:teuthology.orchestra.run.smithi181:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T20:54:58.853 DEBUG:teuthology.orchestra.run.smithi196:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T20:54:58.855 DEBUG:teuthology.orchestra.run.smithi204:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T20:54:58.865 INFO:teuthology.orchestra.run.smithi181.stderr:bash: line 1: ntpq: command not found 2024-05-03T20:54:58.868 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-05-03T20:54:58.869 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-05-03T20:54:58.869 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T20:54:58.869 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2024-05-03T20:54:58.870 INFO:teuthology.orchestra.run.smithi181.stdout:^* hv01.front.sepia.ceph.com 3 6 377 63 +1085us[+1077us] +/- 57ms 2024-05-03T20:54:58.870 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 62 -1351us[-1351us] +/- 65ms 2024-05-03T20:54:58.870 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 63 +210us[ +210us] +/- 81ms 2024-05-03T20:54:58.870 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T20:54:58.872 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T20:54:58.872 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-05-03T20:54:58.872 INFO:teuthology.orchestra.run.smithi196.stdout:^* hv01.front.sepia.ceph.com 3 6 377 61 +1068us[+1071us] +/- 57ms 2024-05-03T20:54:58.872 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 1 -1385us[-1385us] +/- 66ms 2024-05-03T20:54:58.873 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 62 +234us[ +236us] +/- 81ms 2024-05-03T20:54:58.873 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T20:54:58.875 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T20:54:58.875 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-05-03T20:54:58.876 INFO:teuthology.orchestra.run.smithi204.stdout:^* hv01.front.sepia.ceph.com 3 6 377 60 +1020us[+1022us] +/- 57ms 2024-05-03T20:54:58.876 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 60 -1411us[-1409us] +/- 65ms 2024-05-03T20:54:58.876 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 +142us[ +144us] +/- 81ms 2024-05-03T20:54:58.876 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T20:54:58.876 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-03T20:54:58.886 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-03T20:54:58.886 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-03T20:54:58.910 DEBUG:teuthology.orchestra.run.smithi181:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-03T20:54:58.983 DEBUG:teuthology.orchestra.run.smithi196:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-03T20:54:59.049 DEBUG:teuthology.orchestra.run.smithi204:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-03T20:54:59.113 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T20:54:59.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T20:54:59.141 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T20:54:59.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T20:54:59.168 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T20:54:59.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T20:54:59.195 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-03T20:54:59.206 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-03T20:54:59.235 INFO:teuthology.task.internal:Duration was 2213.225674 seconds 2024-05-03T20:54:59.236 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-03T20:54:59.269 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-03T20:54:59.269 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-03T20:54:59.272 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-03T20:54:59.274 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-03T20:54:59.306 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T20:54:59.310 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T20:54:59.311 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T20:54:59.645 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-03T20:54:59.645 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi181.front.sepia.ceph.com 2024-05-03T20:54:59.646 DEBUG:teuthology.orchestra.run.smithi181:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-03T20:54:59.665 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi196.front.sepia.ceph.com 2024-05-03T20:54:59.666 DEBUG:teuthology.orchestra.run.smithi196:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-03T20:54:59.711 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi204.front.sepia.ceph.com 2024-05-03T20:54:59.711 DEBUG:teuthology.orchestra.run.smithi204:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-03T20:54:59.738 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-03T20:54:59.739 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-03T20:54:59.741 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-03T20:54:59.754 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-03T20:54:59.804 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-03T20:54:59.805 DEBUG:teuthology.orchestra.run.smithi181:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-03T20:54:59.807 DEBUG:teuthology.orchestra.run.smithi196:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-03T20:54:59.819 DEBUG:teuthology.orchestra.run.smithi204:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-03T20:55:00.316 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-03T20:55:00.327 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-03T20:55:00.327 DEBUG:teuthology.orchestra.run.smithi181:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-03T20:55:00.350 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-03T20:55:00.375 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-03T20:55:00.399 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-03T20:55:00.410 DEBUG:teuthology.orchestra.run.smithi181:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-03T20:55:00.412 DEBUG:teuthology.orchestra.run.smithi196:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-03T20:55:00.417 DEBUG:teuthology.orchestra.run.smithi204:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-03T20:55:00.432 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = core 2024-05-03T20:55:00.437 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = core 2024-05-03T20:55:00.462 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = core 2024-05-03T20:55:00.493 DEBUG:teuthology.orchestra.run.smithi181:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-03T20:55:00.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T20:55:00.507 DEBUG:teuthology.orchestra.run.smithi196:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-03T20:55:00.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T20:55:00.521 DEBUG:teuthology.orchestra.run.smithi204:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-03T20:55:00.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T20:55:00.547 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-03T20:55:00.562 INFO:teuthology.task.internal:Transferring archived files... 2024-05-03T20:55:00.562 DEBUG:teuthology.misc:Transferring archived files from smithi181:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687487/remote/smithi181 2024-05-03T20:55:00.563 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-03T20:55:00.606 DEBUG:teuthology.misc:Transferring archived files from smithi196:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687487/remote/smithi196 2024-05-03T20:55:00.607 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-03T20:55:00.640 DEBUG:teuthology.misc:Transferring archived files from smithi204:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687487/remote/smithi204 2024-05-03T20:55:00.641 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-03T20:55:00.675 INFO:teuthology.task.internal:Removing archive directory... 2024-05-03T20:55:00.676 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-03T20:55:00.682 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-03T20:55:00.685 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-03T20:55:00.724 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-03T20:55:00.740 INFO:teuthology.task.internal:Not uploading archives. 2024-05-03T20:55:00.740 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-03T20:55:00.762 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-03T20:55:00.762 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-03T20:55:00.764 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-03T20:55:00.767 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-03T20:55:00.776 INFO:teuthology.orchestra.run.smithi181.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 3 20:55 /home/ubuntu/cephtest 2024-05-03T20:55:00.778 INFO:teuthology.orchestra.run.smithi196.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 3 20:55 /home/ubuntu/cephtest 2024-05-03T20:55:00.781 INFO:teuthology.orchestra.run.smithi204.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 3 20:55 /home/ubuntu/cephtest 2024-05-03T20:55:00.782 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-03T20:55:00.796 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-03T20:55:00.862 INFO:teuthology.nuke:Checking targets against current locks 2024-05-03T20:55:00.891 DEBUG:teuthology.nuke:shortname: smithi181 2024-05-03T20:55:00.892 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-03T20:55:00.915 DEBUG:teuthology.nuke:shortname: smithi196 2024-05-03T20:55:00.916 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-03T20:55:00.921 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687487', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-03 20:10:28.696450', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMuJgBR7dwrCRXcE0MRztbywZDTsTI2MxthXnWFQRLNxRkd7MlDYH2IdXqh1FRIkM/xEhx/Uyhvd2vNgohg2qis='} 2024-05-03T20:55:00.939 DEBUG:teuthology.nuke:shortname: smithi204 2024-05-03T20:55:00.939 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-03T20:55:00.946 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687487', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-03 20:10:28.693928', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBpeNGjmmxDguhY9ZoRg+7WOENxhsqR+2Nuq7fIgvxM6U9pn89v6rJgKvx8qlF1a49espqlHJFgKCv9lH4UxUao='} 2024-05-03T20:55:00.962 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687487', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-03 20:10:28.695322', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDudUQ414Z11W7NN34BCo5zkmQMHnmmVNIFbg4FQiR/9i+xblCUsfXiZrHQYSKYjkelN4IHaxmisAWd/hbohSws='} 2024-05-03T20:55:00.972 INFO:teuthology.orchestra.console.smithi181:Power off 2024-05-03T20:55:00.972 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-03T20:55:00.998 DEBUG:teuthology.orchestra.console.smithi181:power off output: Chassis Power Control: Down/Off 2024-05-03T20:55:00.998 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T20:55:01.017 INFO:teuthology.orchestra.console.smithi196:Power off 2024-05-03T20:55:01.018 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-03T20:55:01.030 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2024-05-03T20:55:01.042 DEBUG:teuthology.orchestra.console.smithi196:power off output: Chassis Power Control: Down/Off 2024-05-03T20:55:01.042 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T20:55:01.059 INFO:teuthology.orchestra.console.smithi204:Power off 2024-05-03T20:55:01.060 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-03T20:55:01.071 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-05-03T20:55:01.084 DEBUG:teuthology.orchestra.console.smithi204:power off output: Chassis Power Control: Down/Off 2024-05-03T20:55:01.084 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T20:55:01.108 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2024-05-03T20:55:05.031 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T20:55:05.072 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T20:55:05.109 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T20:55:05.142 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2024-05-03T20:55:05.182 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-05-03T20:55:05.219 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2024-05-03T20:55:09.147 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T20:55:09.183 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T20:55:09.220 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T20:55:09.258 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2024-05-03T20:55:09.294 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is off 2024-05-03T20:55:09.330 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is off 2024-05-03T20:55:09.395 INFO:teuthology.orchestra.console.smithi196:Power off completed 2024-05-03T20:55:09.432 INFO:teuthology.orchestra.console.smithi204:Power off completed 2024-05-03T20:55:13.263 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T20:55:13.373 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off 2024-05-03T20:55:13.474 INFO:teuthology.orchestra.console.smithi181:Power off completed 2024-05-03T20:55:13.580 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/kernel_untar_build}} duration: 2213.225674390793 failure_reason: 'Command failed on smithi181 with status 110: "sudo TESTDIR=/home/ubuntu/cephtest bash -c ''ceph fs subvolumegroup pin cephfs qa random 0.10''"' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=06ebc29797f645da8539fafcabe31f74 status: fail success: false 2024-05-03T20:55:13.580 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-03T20:55:13.698 INFO:teuthology.run:FAIL